diff --git a/docs/development/new_hardware_support.md b/docs/development/new_hardware_support.md index b70e66060d..f53eb24238 100644 --- a/docs/development/new_hardware_support.md +++ b/docs/development/new_hardware_support.md @@ -218,7 +218,7 @@ target/debug/bmc-explorer-cli \ ``` -`--boot-mac` is the MAC address of the interface from which the host is expected to boot. For the common managed-DPU configuration, use the host-facing `pf0` MAC of the primary DPU; for an integrated-NIC configuration, use the NIC selected as the primary boot interface. Use the same MAC for exploration, machine setup, setup status, and boot-order tests. Omit it only when the platform does not require a selected boot interface. See [Boot Interfaces and DPU Modes](../provisioning/boot-interfaces-and-dpu-modes.md) for how NICo selects and persists this value. +`--boot-mac` is the MAC address of the interface from which the host is expected to boot. For the common managed-DPU configuration, use the host-facing `pf0` MAC of the primary DPU; for an integrated-NIC configuration, use the NIC selected as the primary boot interface. Use the same MAC for exploration, machine setup, setup status, and boot-order tests. Omit it only when the platform does not require a selected boot interface. See [Boot Interfaces and DPU Policies](../provisioning/boot-interfaces-and-dpu-modes.md) for how NICo selects and persists this value. `--bmc-port` is the (optional) port on which the BMC listens. HTTPS port 443 is the default; use `--bmc-port ` for a BMC listening elsewhere. @@ -386,4 +386,3 @@ To add one: 1. Wire the variant through `crates/bmc-mock/src/machine_info.rs`: DPU count and type, vendor and product identity, Redfish version, manager, system, chassis, discovery, firmware inventory, and OEM behavior should match the live BMC responses relevant to the test. 1. Add focused tests for the behavior the hardware contribution changes, such as vendor detection, inventory, NIC-mode detection, or provisioning. - diff --git a/docs/glossary.md b/docs/glossary.md index 840bbcf0c7..837494bb43 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -51,6 +51,14 @@ In current deployments, the DPU is a [NVIDIA BlueField-2 or BlueField-3](https:/ The NVIDIA DPU family used by NICo for tenant isolation and site management. A BlueField card has its own ARM complex, BMC, NIC firmware, and OS image. NICo provisions and manages the BlueField side of each ManagedHost before making the Host available to tenants. +### Host DPU Policy + +The operator's desired treatment of DPU hardware on a host. `HostDpuPolicy` has three values: `Manage`, which at the site or resolved level lets NICo provision and attach the DPUs; `UseAsNic`, which asks NICo to run physically present DPU hardware as plain NICs; and `Ignore`, which tells NICo not to configure or attach DPU hardware. For backward compatibility, a per-host `Manage` declaration inherits the site policy rather than overriding it. This policy is intent, not a hardware observation. + +### BlueField Operating Mode + +The mode a BlueField card reports that it is currently running in. `BlueFieldOperatingMode` is either `Dpu` or `Nic`. NICo compares this observed state with the host DPU policy when deciding whether a card needs reconfiguration. Existing protobuf and Redfish boundaries retain the legacy `NicMode` name for compatibility; model code uses `BlueFieldOperatingMode` so observed state is not confused with desired policy. + ## REST API Services and Binaries ### API Server @@ -148,7 +156,7 @@ A VPC virtualization type for tenant instances on zero-DPU hosts. A Flat VPC is ### Zero-DPU Host -A managed host that NICo operates without a NICo-managed DPU — either a host with no DPU hardware (`no_dpu`) or a host whose BlueField DPU is run as a plain NIC (`nic_mode`). NICo does not build an overlay for a zero-DPU host; its tenant instances attach directly to HostInband underlay segments and belong to Flat VPCs. DPU mode is set site-wide or per host in the API server configuration. +A managed host that NICo operates without a NICo-managed DPU — either a host whose DPU policy is `ignore`, or a host whose BlueField DPU is run as a plain NIC through `use_as_nic`. NICo does not build an overlay for a zero-DPU host; its tenant instances attach directly to HostInband underlay segments and belong to Flat VPCs. DPU policy is set site-wide in the API server configuration or per host on its Expected Machine. ### HBN in NICo diff --git a/docs/index.yml b/docs/index.yml index 4e35e54d97..62cff76180 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -112,7 +112,7 @@ navigation: path: provisioning/ingesting-hosts.md - page: Ingesting Hosts (REST API) path: provisioning/ingesting-hosts-rest-api.md - - page: Boot Interfaces and DPU Modes + - page: Boot Interfaces and DPU Policies path: provisioning/boot-interfaces-and-dpu-modes.md - page: Machine Validation path: provisioning/machine-validation.md diff --git a/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-add.md b/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-add.md index eed5657db3..66c9396173 100644 --- a/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-add.md +++ b/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-add.md @@ -16,7 +16,7 @@ nico-admin-cli-expected-machine-add - Add expected machine \[**--host_nics**\] \[**--rack_id**\] \[**--default_pause_ingestion_and_poweron**\] \[**--dpf-enabled**\] \[**--extended**\] \[**--bmc-ip-address**\] -\[**--bmc-retain-credentials**\] \[**--dpu-mode**\] +\[**--bmc-retain-credentials**\] \[**--dpu-policy**\] \[**--disable-lockdown**\] \[**--sort-by**\] \[**-h**\|**--help**\] ## DESCRIPTION @@ -114,23 +114,21 @@ factory-default credentials in Vault as-is\ - false -**--dpu-mode** *\* -Per-host DPU operating mode. \`dpu-mode\` (default): DPUs are managed by -NICo; \`nic-mode\`: DPU hardware present but treated as a plain NIC; -\`no-dpu\`: no DPU hardware at all. Unset defers to the site-wide -\`\[site_explorer\] dpu_mode\` setting (which itself falls back to -\`dpu-mode\` when not set).\ +**--dpu-policy** *\*\ +Per-host DPU policy. \`manage\` (default): inherit the site policy, +which defaults to managing DPUs; \`use-as-nic\`: configure DPU hardware +as plain NICs; \`ignore\`: do not configure or attach DPU hardware. +Unset defers to the site-wide \`\[site_explorer\] dpu_policy\` setting. +The legacy \`--dpu-mode\` flag and values remain accepted.\ \ *Possible values:* -- unspecified +- manage -- dpu-mode +- use-as-nic -- nic-mode - -- no-dpu +- ignore **--disable-lockdown** *\* If true, do not lock down the server as part of lifecycle management @@ -163,7 +161,7 @@ Print help (see a summary with -h) nico-admin-cli expected-machine add --bmc-mac-address 00:11:22:33:44:55 --bmc-username admin --bmc-password mypassword --chassis-serial-number sample_serial-1 nico-admin-cli expected-machine add --bmc-mac-address 00:11:22:33:44:55 --bmc-username admin --bmc-password mypassword --chassis-serial-number sample_serial-1 --meta-name MyMachine --label DATACENTER:XYZ --sku-id DGX-H100-640GB nico-admin-cli expected-machine add --bmc-mac-address 00:11:22:33:44:55 --bmc-username admin --bmc-password mypassword --chassis-serial-number sample_serial-1 --bmc-ip-address 192.0.2.20 -nico-admin-cli expected-machine add --bmc-mac-address 00:11:22:33:44:55 --bmc-username admin --bmc-password mypassword --chassis-serial-number sample_serial-1 --dpu-mode nic-mode +nico-admin-cli expected-machine add --bmc-mac-address 00:11:22:33:44:55 --bmc-username admin --bmc-password mypassword --chassis-serial-number sample_serial-1 --dpu-policy use-as-nic ``` --- diff --git a/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-patch.md b/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-patch.md index 49d10d9fb2..a54b38bfe9 100644 --- a/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-patch.md +++ b/docs/manuals/nico-admin-cli/commands/expected-machine/expected-machine-patch.md @@ -17,7 +17,7 @@ update, preserves unprovided fields). \[**--meta-description**\] \[**--label**\] \[**--sku-id**\] \[**--rack-id**\] \[**--default_pause_ingestion_and_poweron**\] \[**--dpf-enabled**\] \[**--bmc-ip-address**\] \[**--extended**\] -\[**--bmc-retain-credentials**\] \[**--dpu-mode**\] +\[**--bmc-retain-credentials**\] \[**--dpu-policy**\] \[**--disable-lockdown**\] \[**--sort-by**\] \[**-h**\|**--help**\] ## DESCRIPTION @@ -120,22 +120,21 @@ factory-default credentials in Vault as-is\ - false -**--dpu-mode** *\* -Per-host DPU operating mode. \`dpu-mode\` (default): DPUs are managed by -NICo; \`nic-mode\`: DPU hardware present but treated as a plain NIC; -\`no-dpu\`: no DPU hardware at all. Unset preserves the existing -per-host value.\ +**--dpu-policy** *\*\ +Per-host DPU policy. \`manage\`: inherit the site policy, which defaults +to managing DPUs; \`use-as-nic\`: configure DPU hardware as plain NICs; +\`ignore\`: do not configure or attach DPU hardware. Unset preserves the +existing per-host value. The legacy \`--dpu-mode\` flag and values remain +accepted.\ \ *Possible values:* -- unspecified +- manage -- dpu-mode +- use-as-nic -- nic-mode - -- no-dpu +- ignore **--disable-lockdown** *\* If true, do not lock down the server as part of lifecycle management @@ -168,7 +167,7 @@ Print help (see a summary with -h) nico-admin-cli expected-machine patch --bmc-mac-address 00:11:22:33:44:55 --sku-id DGX-H100-640GB nico-admin-cli expected-machine patch --id 12345678-1234-5678-90ab-cdef01234567 --sku-id DGX-H100-640GB nico-admin-cli expected-machine patch --bmc-mac-address 00:11:22:33:44:55 --bmc-username admin --bmc-password mynewpassword -nico-admin-cli expected-machine patch --bmc-mac-address 00:11:22:33:44:55 --dpu-mode no-dpu +nico-admin-cli expected-machine patch --bmc-mac-address 00:11:22:33:44:55 --dpu-policy ignore ``` --- diff --git a/docs/manuals/vpc/flat_vpcs_zero_dpu.md b/docs/manuals/vpc/flat_vpcs_zero_dpu.md index ae572c7be9..ad4b075f4b 100644 --- a/docs/manuals/vpc/flat_vpcs_zero_dpu.md +++ b/docs/manuals/vpc/flat_vpcs_zero_dpu.md @@ -70,7 +70,8 @@ abbreviated as `…/nico/...` thereafter. | Task | Role | Interface | |---|---|---| -| Put hosts in NIC / no-DPU mode (site-wide `dpu_mode`, per-host `ExpectedMachine.dpu_mode`) | Operator | **TOML** — Day 0 / rare; API restart | +| Set hosts to `use_as_nic` or `ignore` through the site-wide `dpu_policy` | Operator | **TOML** — Day 0 / rare; API restart | +| Set one host's `ExpectedMachine.dpu_policy` | Operator | **`nico-admin-cli`** (`em add` / `em patch`) — no restart required | | Declare `HostInband` underlay segments | Operator | **TOML** (`[networks.]`, `type = "hostinband"`) — Day 0 | | Create an additional `HostInband` segment after Day 0 | Operator | **TOML** (`[networks]`) + API restart, or **`nico-admin-cli`** (gRPC `CreateNetworkSegment`) — see [Configuring HostInband Segments](#2-configuring-hostinband-network-segments) | | Inspect / delete a `HostInband` segment | Operator | **`nico-admin-cli`** (`network-segment show` / `delete`) | @@ -99,34 +100,47 @@ are running without a NICo-managed DPU, the underlay segments those hosts sit on are declared as `HostInband` segments, and (optionally) instance types exist so tenants can request the right machines. -### 1. Put hosts in NIC or no-DPU mode +### 1. Set the host DPU policy -Whether a host is a "zero-DPU" host is decided by its **DPU mode**, which has -three values: +Whether NICo treats a host as a "zero-DPU" host is decided by its **DPU policy**, +which has three values: -| `dpu_mode` value | Meaning | +| `dpu_policy` value | Meaning | |---|---| -| `dpu_mode` | *(default)* The DPU is managed by NICo: BFB/firmware upgrades, HBN deployment, DPU agent, and the tenant overlay all apply. | -| `nic_mode` | A DPU is physically present but is operated as a plain NIC. NICo skips DPU provisioning and overlay management for the host. | -| `no_dpu` | The host has no DPU hardware; its NIC sits directly on the underlay. | +| `manage` | *(site/effective default)* The DPU is managed by NICo: BFB/firmware upgrades, HBN deployment, DPU agent, and the tenant overlay all apply. A per-host `manage` declaration inherits the site policy rather than overriding it. | +| `use_as_nic` | A DPU is physically present but should be operated as a plain NIC. NICo skips DPU provisioning and overlay management for the host. | +| `ignore` | NICo does not configure or attach DPU hardware. Use this for a host without DPUs or when installed DPUs should be intentionally ignored. | -Both `nic_mode` and `no_dpu` make the host a **zero-DPU host** for the purposes +Both `use_as_nic` and `ignore` make the host a **zero-DPU host** for the purposes of Flat VPCs: NICo does not manage an overlay for it, and its only valid tenant attachments are `HostInband` segments. -Set the mode in either of two places, with the per-host value taking precedence: +Set the policy in either of two places. Per-host `use_as_nic` and `ignore` +override the site-wide setting; per-host `manage` or an omitted per-host value +instead defers to the site-wide setting for backward compatibility: - **Site-wide**, in the API server configuration: ```toml [site_explorer] - dpu_mode = "nic_mode" # or "no_dpu"; omit entirely for the default "dpu_mode" + dpu_policy = "use_as_nic" # or "ignore"; omit entirely for the default "manage" ``` -- **Per host**, on the host's `ExpectedMachine` entry, via the `dpu_mode` field. - An explicit per-host `nic_mode` or `no_dpu` always wins over the site-wide - setting; a per-host default (or no entry) defers to the site-wide value, which - in turn defaults to managed `dpu_mode`. +- **Per host**, on the host's `ExpectedMachine` entry, via the `dpu_policy` + field. Per-host `use_as_nic` and `ignore` override the site-wide setting; + for backward compatibility, per-host `manage` or an omitted policy defers to + the site-wide value, which defaults to `manage`. + +Existing configuration and admin JSON input remain readable: `dpu_mode` is +accepted as a field alias, and legacy values `dpu_mode`, `nic_mode`, and +`no_dpu` map to `manage`, `use_as_nic`, and `ignore`, respectively. The admin +CLI likewise accepts `--dpu-mode dpu-mode|nic-mode|no-dpu` as legacy aliases; +use `--dpu-policy manage|use-as-nic|ignore` for new automation. + +The Forge protobuf boundary deliberately keeps `ExpectedMachine.dpu_mode` +typed as `DpuMode` for generated-client compatibility. NICo maps `DPU_MODE`, +`NIC_MODE`, and `NO_DPU` immediately to `HostDpuPolicy::Manage`, `UseAsNic`, +and `Ignore`; the model and configuration surfaces use the policy names. Two related Day-0 settings matter for zero-DPU sites: @@ -231,7 +245,7 @@ or `nicocli` (its wrapper), which expose the full surface: For a zero-DPU site, create instance type(s) describing the zero-DPU machines' capabilities and associate those machines, so tenants can request instances of that type. The instance type itself does **not** carry a "no-DPU" flag — what -makes a host zero-DPU is its `dpu_mode` (above). The instance type simply selects +makes a host zero-DPU is its `dpu_policy` (above). The instance type simply selects which machines are allocatable; whether the selected host is zero-DPU then governs the network model at allocation time. @@ -399,9 +413,9 @@ operator SDN integration can tie its switch-side configuration to the VPC. **Operator — site is Flat-ready:** -1. Hosts resolve to a zero-DPU mode. Confirm the intended hosts report - `nic_mode` or `no_dpu` (per-host `ExpectedMachine.dpu_mode`, or the site-wide - `[site_explorer] dpu_mode`). +1. Hosts resolve to a zero-DPU policy. Confirm the intended hosts use + `use_as_nic` or `ignore` (per-host `ExpectedMachine.dpu_policy`, or the + site-wide `[site_explorer] dpu_policy`). 2. `HostInband` segments exist. `nico-admin-cli network-segment show` lists the declared segments with the expected prefix, gateway, and a VLAN/VNI assigned. 3. Each zero-DPU host's data-NIC MACs are registered, and the host shows diff --git a/docs/observability/core_metrics.md b/docs/observability/core_metrics.md index 51ae99e01d..c2ed38cd6c 100644 --- a/docs/observability/core_metrics.md +++ b/docs/observability/core_metrics.md @@ -151,7 +151,7 @@ This file contains a list of metrics exported by NVIDIA Infra Controller (NICo). carbide_site_explorer_create_machines_latency_millisecondshistogramThe time it took to perform create_machines inside site-explorer carbide_site_explorer_created_machines_countgaugeNumber of machine pairs created by Site Explorer after identification carbide_site_explorer_created_power_shelves_countgaugeNumber of power shelves created by Site Explorer after identification -carbide_site_explorer_dpu_migration_signals_countgaugeNumber of DPU NIC-mode migration signals by signal type -- mode-mismatch found, set_nic_mode issued, reset requested, and zero-DPU registered for a NicMode host. +carbide_site_explorer_dpu_migration_signals_countgaugeNumber of DPU NIC-mode migration signals by signal type -- mode-mismatch found, set_nic_mode issued, reset requested, and zero-DPU registered for a host whose DPU policy is use_as_nic. carbide_site_explorer_enabledgaugeWhether site-explorer is enabled (1) or paused (0) carbide_site_explorer_iteration_latency_millisecondshistogramThe time it took to perform one site explorer iteration carbide_site_explorer_last_run_statusgaugeThe status of the latest Site Explorer run diff --git a/docs/provisioning/boot-interfaces-and-dpu-modes.md b/docs/provisioning/boot-interfaces-and-dpu-modes.md index c1362d3c32..27947921b5 100644 --- a/docs/provisioning/boot-interfaces-and-dpu-modes.md +++ b/docs/provisioning/boot-interfaces-and-dpu-modes.md @@ -1,10 +1,10 @@ -# Boot Interfaces and DPU Modes +# Boot Interfaces and DPU Policies -This guide explains how NICo decides **which interface a host boots from**, how a host's **DPUs are managed**, and how operators configure both through the Expected Machines table. It is the deep companion to [Ingesting Hosts](ingesting-hosts.md): that page covers the end-to-end ingest flow and the basic `expected_machines.json`; this page covers the per-host and per-NIC knobs (`dpu_mode`, `host_nics`), **what the defaults do when you set nothing**, and how a boot device is chosen and applied behind the scenes. +This guide explains how NICo decides **which interface a host boots from**, how a host's **DPUs are managed**, and how operators configure both through the Expected Machines table. It is the deep companion to [Ingesting Hosts](ingesting-hosts.md): that page covers the end-to-end ingest flow and the basic `expected_machines.json`; this page covers the per-host and per-NIC knobs (`dpu_policy`, `host_nics`), **what the defaults do when you set nothing**, and how a boot device is chosen and applied behind the scenes. For the DHCP and network-segment substrate these knobs sit on (how a relay's `giaddr` maps to a segment), see [IP and Network Configuration](ip-and-network-configuration.md). -> **Who should read this.** Operators configuring hosts for ingestion, and anyone debugging "why did this host boot from *that* interface?" **Most hosts need no configuration here** — the defaults handle the common managed-DPU case. Reach for the knobs in [Section 2](#2-configuring-via-expected-machines-and-the-defaults) and [Section 3](#3-scenarios) only for NIC-mode, zero-DPU, or integrated-NIC hosts. [Sections 6](#6-behind-the-scenes-how-a-boot-device-is-chosen-and-set)–[7](#7-the-boot-interface-data-model) explain the machinery when you need to trace a problem. +> **Who should read this.** Operators configuring hosts for ingestion, and anyone debugging "why did this host boot from *that* interface?" **Most hosts need no configuration here** — the defaults handle the common managed-DPU case. Reach for the knobs in [Section 2](#2-configuring-via-expected-machines-and-the-defaults) and [Section 3](#3-scenarios) only for `use_as_nic`, `ignore`, or integrated-NIC hosts. [Sections 6](#6-behind-the-scenes-how-a-boot-device-is-chosen-and-set)–[7](#7-the-boot-interface-data-model) explain the machinery when you need to trace a problem. --- @@ -14,11 +14,17 @@ Historically, two separate decisions were conflated into "what kind of host is t | Axis | Question | Controlled by | |---|---|---| -| **DPU management** | Does NICo manage this host's DPUs (upgrade them, run agents, serve the host's admin network over the DPU overlay)? | `dpu_mode` | +| **DPU management** | Does NICo manage this host's DPUs (upgrade them, run agents, serve the host's admin network over the DPU overlay)? | `dpu_policy` | | **Boot interface** | Which NIC does the host OS actually boot from and run its management network on? | the host's **primary** interface | A "normal" host couples them (managed DPU + boot through that DPU). But they are independent: you can keep a host's DPUs **managed** and still boot it from a plain **integrated NIC**. This guide treats the two axes separately, because the configuration knobs are separate. +The policy is also separate from the card's current hardware state. NICo models +operator intent as `HostDpuPolicy::{Manage, UseAsNic, Ignore}` and the mode +reported by a BlueField as `BlueFieldOperatingMode::{Dpu, Nic}`. Existing +protobuf and Redfish boundaries retain the legacy `NicMode` name for +compatibility, but model code does not use that name for observed state. + ### Network segment types A host's management network lives on one of a few segment types. Which one depends on **how the host boots**: @@ -44,23 +50,27 @@ Boot and DPU configuration is **declarative**: you describe the host in the Expe ### If you set nothing (the default) -**Most hosts need zero boot/DPU configuration.** With no `dpu_mode` and no `host_nics`: +**Most hosts need zero boot/DPU configuration.** When neither the host nor the site sets `dpu_policy`, and the host has no `host_nics` declaration: -- `dpu_mode` defaults to **`dpu_mode`** (managed) — NICo ingests and manages the host's DPUs, and the host boots through its primary DPU on the **Admin** network. +- The effective `dpu_policy` resolves to **`manage`** — NICo ingests and manages the host's DPUs, and the host boots through its primary DPU on the **Admin** network. - Site-explorer **auto-selects the boot interface**: the lowest-PCI DPU host-PF (the NIC a DPU presents to the host). - The host's IP comes from whichever segment its DHCP relay lands in (see [IP and Network Configuration](ip-and-network-configuration.md)). So a standard DPU host is handled entirely by defaults. The knobs below exist for the hosts that *don't* fit that mold. -### `dpu_mode` +### `dpu_policy` -| Value (JSON / CLI) | Meaning | -|---|---| -| `dpu_mode` / `dpu-mode` (default) | DPUs are managed by NICo; the host boots through its primary DPU on the Admin overlay. | -| `nic_mode` / `nic-mode` | DPU hardware is present but treated as a **plain NIC**. Site-explorer explores it but does **not** link or manage it; the host boots on **HostInband**. | -| `no_dpu` / `no-dpu` | No DPU hardware at all — a plain host NIC on **HostInband**. DPU exploration is skipped entirely. | +| JSON/TOML value | CLI value | Meaning | +|---|---|---| +| `manage` (site/effective default) | `manage` | As a site or resolved policy, DPUs are managed by NICo and the host boots through its primary DPU on the Admin overlay. A per-host `manage` declaration inherits the site policy rather than overriding it. | +| `use_as_nic` | `use-as-nic` | DPU hardware is present but should operate as a **plain NIC**. Site-explorer explores it but does **not** link or manage it; the host boots on **HostInband**. | +| `ignore` | `ignore` | NICo does not configure or attach DPU hardware. Use this for a host without DPUs or when installed DPUs should be intentionally ignored; the host boots through a plain NIC on **HostInband**. | + +**Resolution order:** per-host `use_as_nic` and `ignore` policies override the site. For backward compatibility, per-host `manage`—like an omitted per-host policy—defers to the site-wide `[site_explorer] dpu_policy`; if the site policy is also unset, the result is `manage`. + +Legacy declarations remain accepted when deserializing configuration and admin JSON input: the field alias `dpu_mode` and values `dpu_mode`, `nic_mode`, and `no_dpu` map to `manage`, `use_as_nic`, and `ignore`, respectively. The admin CLI likewise accepts the legacy `--dpu-mode dpu-mode|nic-mode|no-dpu` forms, but new automation should use `--dpu-policy manage|use-as-nic|ignore`. -**Resolution order:** a per-host `dpu_mode` on the Expected Machine wins; if unset, the site-wide `[site_explorer] dpu_mode` setting applies; if that too is unset, the default is `dpu_mode`. +The Forge protobuf boundary deliberately keeps `ExpectedMachine.dpu_mode` typed as `DpuMode` so existing generated clients remain compatible. NICo maps its `DPU_MODE`, `NIC_MODE`, and `NO_DPU` values immediately to `HostDpuPolicy::Manage`, `UseAsNic`, and `Ignore`; model and configuration code use only the policy-oriented names. ### `host_nics` (per-NIC declaration) @@ -84,7 +94,7 @@ The optional `host_nics` array declares specifics for individual host NICs. Each "bmc_username": "root", "bmc_password": "default-password1", "chassis_serial_number": "SERIAL-1", - "dpu_mode": "dpu_mode", + "dpu_policy": "manage", "host_nics": [ { "mac_address": "C4:5A:B1:C8:38:10", @@ -102,7 +112,7 @@ nico-admin-cli -a em add \ --bmc-mac-address C4:5A:B1:C8:38:0D \ --bmc-username root --bmc-password default-password1 \ --chassis-serial-number SERIAL-1 \ - --dpu-mode dpu-mode \ + --dpu-policy manage \ --host_nics '[{"mac_address":"C4:5A:B1:C8:38:10","primary":true,"network_segment_type":"host_inband"}]' ``` @@ -116,15 +126,15 @@ Concrete recipes for the cases beyond the default. All assume the rest of the Ex ### 3.1 Standard DPU host -Nothing to configure. Managed DPUs, boot through the primary DPU on Admin. This is the default. +With the site policy unset or set to `manage`, there is nothing to configure. DPUs are managed and the host boots through the primary DPU on Admin. ### 3.2 Zero-DPU host (no DPU hardware) -A plain server with one or more host NICs and no DPU. Declare `no_dpu` and mark the boot NIC primary: +A plain server with one or more host NICs and no DPU. Declare `ignore` and mark the boot NIC primary: ```json { - "dpu_mode": "no_dpu", + "dpu_policy": "ignore", "host_nics": [ { "mac_address": "AA:BB:CC:00:00:10", "primary": true, "network_segment_type": "host_inband" } ] @@ -135,11 +145,11 @@ The host boots from that NIC on HostInband and gets its IP from central NICo DHC ### 3.3 DPU in NIC mode -The host has DPU hardware, but you want it treated as a plain NIC (not managed). Declare `nic_mode`. Site-explorer still explores the DPU (and will issue the mode flip — see [3.5](#35-flipping-a-dpu-to-nic-mode)) but does not link it as a managed machine; the host boots HostInband: +The host has DPU hardware, but you want it treated as a plain NIC (not managed). Declare `use_as_nic`. Site-explorer still explores the DPU (and will issue the physical mode flip — see [3.5](#35-flipping-a-dpu-to-nic-mode)) but does not link it as a managed machine; the host boots HostInband: ```json { - "dpu_mode": "nic_mode", + "dpu_policy": "use_as_nic", "host_nics": [ { "mac_address": "AA:BB:CC:00:00:20", "primary": true, "network_segment_type": "host_inband" } ] @@ -148,11 +158,11 @@ The host has DPU hardware, but you want it treated as a plain NIC (not managed). ### 3.4 Boot an integrated NIC while keeping the DPUs managed -This is the case where the two axes genuinely diverge: the host has cabled, explorable DPUs you **want managed** (for the data plane), but you want the host OS to boot from a **plain integrated NIC** rather than through a DPU. Declare `dpu_mode` (managed) *and* mark the integrated NIC primary on HostInband: +This is the case where the two axes genuinely diverge: the host has cabled, explorable DPUs you **want managed** (for the data plane), but you want the host OS to boot from a **plain integrated NIC** rather than through a DPU. First ensure the site-wide policy is unset or `manage`; a per-host `manage` declaration inherits the site and cannot override `use_as_nic` or `ignore`. Then leave the host policy unset (or explicitly set `manage`) and mark the integrated NIC primary on HostInband: ```json { - "dpu_mode": "dpu_mode", + "dpu_policy": "manage", "host_nics": [ { "mac_address": "AA:BB:CC:00:00:30", "primary": true, "network_segment_type": "host_inband" } ] @@ -161,14 +171,14 @@ This is the case where the two axes genuinely diverge: the host has cabled, expl NICo keeps the DPUs explored, linked, and underlay-addressed (running agents for the data plane), but the host boots from the integrated NIC. The DPU-backed admin links are kept but go **dormant** — the host's admin/boot path is the HostInband NIC. -> Previously this required faking `no_dpu`/`nic_mode`, which threw away DPU management to get integrated boot. The two are now decoupled. +> Previously this required selecting an unmanaged-DPU policy, which threw away DPU management to get integrated boot. The two are now decoupled. ### 3.5 Flipping a DPU to NIC mode -To change a host that's already ingested (e.g. from managed-DPU to NIC mode), update its Expected Machine `dpu_mode`, then force-delete and let it re-ingest so site-explorer re-explores and applies the new mode: +To change a host that's already ingested (e.g. from managed-DPU to NIC mode), update its Expected Machine `dpu_policy`, then force-delete and let it re-ingest so site-explorer re-explores and applies the new physical mode: ```bash -nico-admin-cli -a em patch --bmc-mac-address --dpu-mode nic-mode +nico-admin-cli -a em patch --bmc-mac-address --dpu-policy use-as-nic nico-admin-cli -a machine force-delete --machine --delete-interfaces ``` @@ -184,10 +194,10 @@ All of these are **admin-only**; the Forge gRPC service enforces admin authoriza | admin-cli | Forge RPC | Purpose | |---|---|---| -| `em add …` | `AddExpectedMachine` | Add one host (BMC creds, `--dpu-mode`, `--host_nics`, metadata). | +| `em add …` | `AddExpectedMachine` | Add one host (BMC creds, `--dpu-policy`, `--host_nics`, metadata). | | `em show [--bmc-mac-address ]` | `GetAllExpectedMachines` / `GetExpectedMachine` | List all, or show one. Add `-f json` to export. | | `em update --filename ` | `UpdateExpectedMachine` | Full replacement of one entry from JSON. | -| `em patch --bmc-mac-address …` | `UpdateExpectedMachine` | Partial update (e.g. `--dpu-mode`), preserving other fields. | +| `em patch --bmc-mac-address …` | `UpdateExpectedMachine` | Partial update (e.g. `--dpu-policy`), preserving other fields. | | `em delete --bmc-mac-address ` | `DeleteExpectedMachine` | Remove one entry. | | `em replace-all --filename ` | (bulk) | Replace the entire table from a file. | | `em erase` | (bulk) | Erase the entire table. | @@ -216,7 +226,7 @@ All of these are **admin-only**; the Forge gRPC service enforces admin authoriza ## 5. Web UI -The NICo admin web UI (`/admin/…`) is primarily for **visibility**, with a focused set of boot/ingestion actions. **There is no DPU-mode-switching control in the UI** — change `dpu_mode` via Expected Machines (CLI/JSON) as in [Section 2](#2-configuring-via-expected-machines-and-the-defaults). +The NICo admin web UI (`/admin/…`) is primarily for **visibility**, with a focused set of boot/ingestion actions. **There is no DPU-policy control in the UI** — change `dpu_policy` via Expected Machines (CLI/JSON) as in [Section 2](#2-configuring-via-expected-machines-and-the-defaults). **View:** @@ -263,7 +273,7 @@ At each boot-config step the controller resolves the target via `load_boot_predi 1. The host's **own owned interface row** (`machine_interfaces`), if it has a boot interface — used once the host has taken its first DHCP lease. 2. Otherwise, the **boot prediction** (`pick_boot_prediction`) — used *before* the first lease. 3. Otherwise, a classification: - - **AwaitingNic** — a zero-DPU/NIC-mode host whose boot NIC hasn't appeared yet; wait. + - **AwaitingNic** — an `ignore`/`use_as_nic` host whose boot NIC hasn't appeared yet; wait. - **Missing** — a host that *should* have a boot interface (it has DPUs) but doesn't; a fault to investigate. > **Key timing.** A host has no `machine_interfaces` row until its first DHCP lease. Predictions are what let the controller configure boot **before** that lease. Once the host leases and the prediction is promoted to an owned row, the **owned row supersedes** the prediction. @@ -272,7 +282,7 @@ At each boot-config step the controller resolves the target via `load_boot_predi - `configure_host_bios` (at `WaitingForPlatformConfiguration`) calls Redfish `machine_setup` with the resolved boot interface; on Dell this schedules a BIOS job (`WaitingForBiosJob`). - `PollingBiosSetup` verifies the BIOS settings took. -- `SetBootOrder` sets the host boot order via Redfish — **DPU-first** for DPU hosts; for zero-DPU/NIC-mode hosts it targets the resolved HostInband interface (a "no DPU" response from the BMC is expected and treated as success). +- `SetBootOrder` sets the host boot order via Redfish — **DPU-first** for managed-DPU hosts; for `ignore`/`use_as_nic` hosts it targets the resolved HostInband interface (a "no DPU" response from the BMC is expected and treated as success). - On a reprovision repair, `check_host_boot_config` re-checks BIOS + boot order and only remediates if they drifted. --- @@ -334,7 +344,7 @@ The interfaces section shows each NIC's MAC, segment, and which one is `primary` | Symptom | Likely cause / action | |---|---| | `boot_interface_mac_mismatch` (pairing blocker) | The host's boot MAC doesn't match any discovered DPU's pf0 MAC. Expected for an integrated-NIC host — declare the integrated NIC `primary` (see [3.4](#34-boot-an-integrated-nic-while-keeping-the-dpus-managed)); otherwise check the exploration reports. See [Ingesting Hosts → pairing blockers](ingesting-hosts.md#common-blockers-during-host--dpu-pairing). | -| Host stuck waiting for a boot NIC | A zero-DPU/NIC-mode host whose boot NIC hasn't leased yet (`AwaitingNic`). Confirm the NIC is cabled and DHCP-reachable on its HostInband segment. | +| Host stuck waiting for a boot NIC | An `ignore`/`use_as_nic` host whose boot NIC hasn't leased yet (`AwaitingNic`). Confirm the NIC is cabled and DHCP-reachable on its HostInband segment. | | Boot interface wrong after a DPU↔NIC-mode flip | Use **Restore Boot Interface** in the web UI, or re-ingest ([3.5](#35-flipping-a-dpu-to-nic-mode)). | | DPU mode "unknown" (`dpu_nic_mode_unknown`) | DPU BMC firmware too old to report mode. Install a fresh DPU OS — see [Ingesting Hosts](ingesting-hosts.md#dpu-related-issues-installing-a-fresh-dpu-os). | diff --git a/docs/provisioning/ingesting-hosts.md b/docs/provisioning/ingesting-hosts.md index 0a7b0b69f1..36d3d994af 100644 --- a/docs/provisioning/ingesting-hosts.md +++ b/docs/provisioning/ingesting-hosts.md @@ -189,7 +189,7 @@ Each entry supports additional optional fields: ``` - **`dpf_enabled`** (bool): Enable/disable DPF for this host. -- **`dpu_mode`** (`"dpu_mode"` | `"nic_mode"` | `"no_dpu"`): Per-host DPU operating mode. +- **`dpu_policy`** (`"manage"` | `"use_as_nic"` | `"ignore"`): Per-host policy for managing DPU hardware. `use_as_nic` and `ignore` override the site policy; for backward compatibility, `manage` or an omitted field inherits the site-wide policy, which defaults to `manage`. Existing manifests using the `dpu_mode` field and `"dpu_mode"` | `"nic_mode"` | `"no_dpu"` values remain accepted as deserialization aliases. - **`bmc_retain_credentials`** (bool): Skip BMC password rotation. - **`default_pause_ingestion_and_poweron`** (bool): Pause ingestion and power-on for this host. - **`bmc_ip_address`** (string): Static BMC IP (pre-allocates a machine interface). @@ -327,4 +327,4 @@ Export the current table as JSON: ```bash nico-admin-cli -a -f json em show -``` \ No newline at end of file +```