-
Notifications
You must be signed in to change notification settings - Fork 150
docs: Document site setup API parity #3462
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,31 +15,27 @@ instance lifecycle.** | |
|
|
||
| The following steps can be used to reboot a machine: | ||
|
|
||
| ### 1. Obtain access to `nico-admin-cli` | ||
| ### 1. Obtain access to `nicocli` | ||
|
|
||
| See [nico-admin-cli access on a NICo deployment](nico_admin_cli.md). | ||
| Configure `nicocli` for the target REST API. The caller's organization must have an Infrastructure Provider that owns the Site containing the Machine, and the caller must have the `PROVIDER_ADMIN` role. | ||
|
|
||
| ### 2. Execute the `nico-admin-cli machine reboot` command | ||
| ### 2. Execute the Machine power control operation | ||
|
|
||
| `nico-admin-cli machine reboot` can be used to restart a machine. | ||
| It always will require the machine's BMC IP and port to be specified. | ||
|
|
||
| BMC credentials can either be explicitly passed, or the `--machine-id` parameter | ||
| can be used to let the NICo site controller read the last known credentials | ||
| for the machine. | ||
|
|
||
| Rebooting a machine will also always reset its boot order. The machine | ||
| will PXE boot, and thereby will be able to retrieve new boot instructions from | ||
| the NICo site controller. | ||
|
|
||
| **Example:** | ||
| Use `GracefulRestart` when the operating system can shut down cleanly. Use `ForceRestart` only when a graceful restart is not possible. | ||
|
|
||
| ```bash | ||
| /opt/nico/nico-admin-cli -a https://127.0.0.1:1079 machine reboot --address 123.123.123.123 --port 9999 --machine-id="60cef902-9779-4666-8362-c9bb4b37184f" | ||
| nicocli machine power-control-machine machine-power-control-machine \ | ||
| --action GracefulRestart \ | ||
| <machine-id> | ||
| ``` | ||
|
|
||
| or using username and password: | ||
| If the Machine has an attached Instance, acknowledge the workload disruption explicitly: | ||
|
|
||
| ```bash | ||
| /opt/nico/nico-admin-cli -a https://127.0.0.1:1079 machine reboot --address 123.123.123.123 --port 9999 --username myhost --password mypassword | ||
| nicocli machine power-control-machine machine-power-control-machine \ | ||
| --action GracefulRestart \ | ||
| --acknowledge-attached-instance true \ | ||
| <machine-id> | ||
| ``` | ||
|
|
||
| A successful request returns HTTP 202. Retrieve the Machine afterward with `nicocli machine get <machine-id>` and confirm that it returns to the expected lifecycle state. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
# Inspect the target document around the cited line
wc -l docs/playbooks/machine_reboot.md
sed -n '1,120p' docs/playbooks/machine_reboot.md
# Locate the referenced API/docs mention in the repository, if any
rg -n "power-control|HTTP 202|machine get|poll|polling|lifecycle state" docs -SRepository: NVIDIA/infra-controller Length of output: 19463 🏁 Script executed: #!/bin/bash
set -euo pipefail
wc -l docs/playbooks/machine_reboot.md
sed -n '1,120p' docs/playbooks/machine_reboot.md
rg -n "power-control|HTTP 202|machine get|poll|polling|lifecycle state" docs -SRepository: NVIDIA/infra-controller Length of output: 19463 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Find any API reference or runbook text for the machine power-control endpoint
rg -n "power-control-machine|GracefulRestart|ForceRestart|HTTP 202|accepted" docs -S
# Read the nearby sections in the machine reboot playbook and any lifecycle guidance
sed -n '1,120p' docs/playbooks/machine_reboot.md
sed -n '1,220p' docs/overview/lifecycle.md
sed -n '1,220p' docs/architecture/state_handling.mdRepository: NVIDIA/infra-controller Length of output: 20670 🌐 Web query:
💡 Result: In the NVIDIA Infra Controller (NICo) architecture, power control operations—such as those managed via the machine power control endpoints—are typically handled asynchronously, returning an HTTP 202 Accepted status [1]. This status indicates that the request has been received and queued for processing, but the operation has not yet completed [2]. Because the state handling is asynchronous and designed for reliability, resources undergo lifecycle state transitions that may take time or require periodic retries by the controller [2]. NICo utilizes a state controller mechanism where handler functions are scheduled for execution periodically (typically every 30 seconds) to ensure resources reach their desired state, even in the event of intermittent failures [2]. To track the progress of these power control operations or determine the current state of a machine, users should: 1. Lifecycle State Polling: Instead of expecting an immediate result from the power control request, the system relies on the resource's lifecycle state machine [2][3][4]. Users can poll the specific machine's status via the REST API to observe transitions as the controller processes the request [5][6]. 2. Observability Metrics: NICo provides Prometheus metrics that expose detailed lifecycle timestamps, current states, and expected next transitions [7]. These metrics can be used to monitor whether a machine is stuck in a state or has successfully completed a transition (e.g., from a power-off state to an operational state) [7]. 3. Task/Status Tracking: For certain operations, the API provides task-based tracking or status history, allowing users to verify the completion of requested actions [5][6]. In summary, an HTTP 202 response signifies that the intent to change the machine's power state has been successfully enqueued [2]. Subsequent verification of the operation's success should be performed by monitoring the resource's lifecycle state or associated status metrics through the NICo REST API [5][6][7]. Citations:
Poll until the reboot reaches a terminal state. HTTP 202 only means the request was accepted; a single 🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # Site Setup API Parity | ||
|
|
||
| Use the REST API and `nicocli` for site setup whenever an operation is listed as covered below. Direct `nico-admin-cli` access remains necessary only for the gaps that have not yet reached the REST API. | ||
|
|
||
| | Site setup task | Current status | Preferred command or tracked gap | | ||
| |---|---|---| | ||
| | Store the site-wide BMC root credential | Covered | `nicocli bmc-credential create --site-id <site-uuid> --kind SiteWideRoot --password <password>` | | ||
| | Store the site-default host UEFI credential | Covered | `nicocli uefi-credential create --site-id <site-uuid> --kind Host --password <password>` | | ||
| | Store the site-default DPU UEFI credential | Covered | `nicocli uefi-credential create --site-id <site-uuid> --kind DPU --password <password>` | | ||
| | Register an Expected Machine | Covered | `nicocli expected-machine create --site-id <site-uuid> --bmc-mac-address <mac> --chassis-serial-number <serial> --default-bmc-username <user> --default-bmc-password <password>` | | ||
| | Register Expected Machines in a batch | Covered | `nicocli expected-machine batch-create --data-file expected-machines.json` | | ||
|
Comment on lines
+7
to
+11
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: rg -n --hidden --glob '!**/.git/**' -S "bmc-credential create|uefi-credential create|expected-machine create|--password|password.*stdin|prompt|secret-file|read-password|password-file|nicocli" docs .Repository: NVIDIA/infra-controller Length of output: 50380 🏁 Script executed: sed -n '1,40p' docs/provisioning/site-setup-api-parity.md
printf '\n---\n'
sed -n '110,170p' docs/provisioning/ingesting-hosts.md
printf '\n---\n'
sed -n '180,210p' docs/manuals/nicocli-reference.mdRepository: NVIDIA/infra-controller Length of output: 6460 🏁 Script executed: sed -n '35,80p' docs/provisioning/ingesting-hosts-rest-api.md
printf '\n---\n'
sed -n '430,460p' docs/manuals/nicocli-reference.md
printf '\n---\n'
sed -n '1,80p' docs/manuals/nico-admin-cli/commands/credential/credential-add-bmc.md
printf '\n---\n'
sed -n '1,80p' docs/manuals/nico-admin-cli/commands/credential/credential-add-uefi.mdRepository: NVIDIA/infra-controller Length of output: 6350 Stop documenting real secrets as command-line arguments. These examples expose BMC and UEFI passwords in shell history and process listings. Use a supported prompt/stdin/secret-file path if the CLI has one; otherwise replace the examples with an approved ephemeral invocation pattern and avoid
📍 Affects 2 files
🤖 Prompt for AI AgentsSource: Path instructions |
||
| | Approve, list, or remove measured-boot machine trust rules | Gap | Tracked by [#2801](https://github.com/NVIDIA/infra-controller/issues/2801). Use `nico-admin-cli attestation measured-boot site trusted-machine approve`, `list`, or `remove` until that issue is complete. | | ||
| | Approve, list, or remove measured-boot profile trust rules | Gap | Tracked by [#2801](https://github.com/NVIDIA/infra-controller/issues/2801). Use `nico-admin-cli attestation measured-boot site trusted-profile approve`, `list`, or `remove` until that issue is complete. | | ||
| | Clear a Site Explorer endpoint error | Gap | Tracked by [#2802](https://github.com/NVIDIA/infra-controller/issues/2802). Use `nico-admin-cli site-explorer clear-error <bmc-ip>` until that issue is complete. | | ||
| | Queue a Site Explorer endpoint for re-exploration | Gap | Tracked by [#2802](https://github.com/NVIDIA/infra-controller/issues/2802). Use `nico-admin-cli site-explorer re-explore <bmc-ip>` until that issue is complete. Bulk selection and execution also remain part of this gap. | | ||
|
Comment on lines
+12
to
+15
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== site-setup-api-parity.md ==\n'
nl -ba docs/provisioning/site-setup-api-parity.md | sed -n '1,80p'
printf '\n== ingesting-hosts.md (around lines 100-120) ==\n'
nl -ba docs/provisioning/ingesting-hosts.md | sed -n '100,120p'
printf '\n== ingesting-hosts.md (around lines 238-255) ==\n'
nl -ba docs/provisioning/ingesting-hosts.md | sed -n '238,255p'Repository: NVIDIA/infra-controller Length of output: 233 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== site-setup-api-parity.md (1-80) ==\n'
sed -n '1,80p' docs/provisioning/site-setup-api-parity.md | cat -n
printf '\n== ingesting-hosts.md (100-120) ==\n'
sed -n '100,120p' docs/provisioning/ingesting-hosts.md | cat -n
printf '\n== ingesting-hosts.md (238-255) ==\n'
sed -n '238,255p' docs/provisioning/ingesting-hosts.md | cat -nRepository: NVIDIA/infra-controller Length of output: 6169 Include the 🤖 Prompt for AI AgentsSource: Path instructions |
||
|
|
||
| ## Remaining parity plan | ||
|
|
||
| The REST/nicocli parity work is tracked under [#2852](https://github.com/NVIDIA/infra-controller/issues/2852): | ||
|
|
||
| - [#2799](https://github.com/NVIDIA/infra-controller/issues/2799) covers site-wide BMC and UEFI credentials and is complete. | ||
| - [#2801](https://github.com/NVIDIA/infra-controller/issues/2801) adds measured-boot trust approval operations. | ||
| - [#2802](https://github.com/NVIDIA/infra-controller/issues/2802) adds Site Explorer clear-error and re-explore operations. | ||
| - [#2803](https://github.com/NVIDIA/infra-controller/issues/2803) covers additional admin-cli inventory. Its implementation merged in [#3241](https://github.com/NVIDIA/infra-controller/pull/3241); the issue remains open for administrative closure. | ||
|
|
||
| After #2801 and #2802 are complete, update this table to replace the remaining direct admin-cli commands with their generated `nicocli` commands. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: NVIDIA/infra-controller
Length of output: 50383
🏁 Script executed:
Repository: NVIDIA/infra-controller
Length of output: 16588
🏁 Script executed:
Repository: NVIDIA/infra-controller
Length of output: 1165
🏁 Script executed:
Repository: NVIDIA/infra-controller
Length of output: 50379
🏁 Script executed:
Repository: NVIDIA/infra-controller
Length of output: 26807
Remove the repeated
machine-power-control-machinetoken. Both snippets should readnicocli machine power-control-machine --action GracefulRestart <machine-id>.🤖 Prompt for AI Agents