Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,15 +233,15 @@ Runtime Intelligence smoke:
| Decision owner | `Lab remains the final deployment decision owner` | EdgeEnv, AIGuard, Orchestrator는 evidence provider이고 최종 판단은 Lab이 소유합니다. |
| EdgeEnv regression gate | EdgeEnv comparability / regression evidence | runtime regression은 EdgeEnv comparability context가 있을 때만 해석합니다. |
| Telemetry/replay quality | telemetry replay gap, `runtime_history_seed_run_config_traceability` | Runtime history seed와 `run_config` traceability가 보존됐는지 확인합니다. |
| Operation context | `Orchestrator queue/deadline/fallback markers` | queue pressure, `max_total_queue_depth`, deadline miss, fallback count를 한눈에 묶습니다. |
| Operation context | `Reviewer operation quick scan`, `Orchestrator queue/deadline/fallback markers` | queue pressure, `max_total_queue_depth`, deadline miss, fallback count, Jetson/device-local preservation identity를 한눈에 묶습니다. |
| AIGuard warnings | deterministic AIGuard runtime operation evidence | AIGuard warning은 Lab policy를 덮어쓰지 않는 review evidence입니다. |

Marker group:

| 그룹 | 핵심 row / label | 이유 |
|---|---|---|
| Producer lineage | `edgeenv_orchestrator_producer_lineage`, `runtime_history_seed_run_config_traceability` | EdgeEnv/Orchestrator lineage가 AIGuard와 Lab까지 보존됐는지 확인합니다. |
| Queue pressure | `Orchestrator queue/deadline/fallback markers`, `AIGuard max queue raw-context traceability` | `max_total_queue_depth`가 AIGuard deterministic raw context와 연결되는지 보여줍니다. |
| Queue pressure | `Reviewer operation quick scan`, `Orchestrator queue/deadline/fallback markers`, `AIGuard max queue raw-context traceability` | `max_total_queue_depth`가 AIGuard deterministic raw context와 연결되는지 보여주고 device-local preservation label을 쉽게 찾게 합니다. |
| Replay / preservation | `Runtime replay duration scope`, `Lab EdgeEnv preservation context`, `Jetson/device-local EdgeEnv preservation run`, `Jetson/device-local EdgeEnv preservation details` | replay duration과 `identity=jetson_device_local_preservation`, `path=device_local_starter` label을 빠르게 찾게 합니다. |
| Task / operation risk | `Orchestrator task event rollup`, `AIGuard task event rollup evidence`, `AIGuard runtime operation anomalies` | scheduler delay, deadline miss, fallback decision, queue/drop reason을 review context로 보여줍니다. |
| Remote starter boundary | `AIGuard remote dispatch event summary`, `Remote fallback starter evidence`, `production_remote_execution=false` | remote dispatch를 production execution이 아니라 starter evidence로 제한합니다. |
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -527,15 +527,15 @@ areas before opening detailed evidence rows.
| Decision owner | `Lab remains the final deployment decision owner` | The report is Lab-owned; EdgeEnv, AIGuard, and Orchestrator provide evidence only. |
| EdgeEnv regression gate | EdgeEnv comparability / regression evidence | Runtime regression is interpreted only after EdgeEnv comparability context is present. |
| Telemetry/replay quality | telemetry replay gaps and `runtime_history_seed_run_config_traceability` | Shows whether Runtime history seed and `run_config` traceability are available. |
| Operation context | `Orchestrator queue/deadline/fallback markers` | Groups queue pressure, `max_total_queue_depth`, deadline misses, and fallback count. |
| Operation context | `Reviewer operation quick scan`, `Orchestrator queue/deadline/fallback markers` | Groups queue pressure, `max_total_queue_depth`, deadline misses, fallback count, and Jetson/device-local preservation identity. |
| AIGuard warnings | deterministic AIGuard runtime operation evidence | Preserves warning evidence without overriding Lab deployment policy. |

Reviewer marker map:

| Marker group | Key rows / labels | Why it matters |
|---|---|---|
| Producer lineage | `edgeenv_orchestrator_producer_lineage`, `runtime_history_seed_run_config_traceability` | Confirms EdgeEnv/Orchestrator lineage reached AIGuard and Lab. |
| Queue pressure | `Orchestrator queue/deadline/fallback markers`, `AIGuard max queue raw-context traceability` | Makes `max_total_queue_depth` traceable back to deterministic AIGuard raw context. |
| Queue pressure | `Reviewer operation quick scan`, `Orchestrator queue/deadline/fallback markers`, `AIGuard max queue raw-context traceability` | Makes `max_total_queue_depth` traceable back to deterministic AIGuard raw context while keeping device-local preservation labels easy to spot. |
| Replay and preservation | `Runtime replay duration scope`, `Lab EdgeEnv preservation context`, `Jetson/device-local EdgeEnv preservation run`, `Jetson/device-local EdgeEnv preservation details` | Shows replay duration, Lab preservation, and `identity=jetson_device_local_preservation` / `path=device_local_starter` labels. |
| Task and operation risk | `Orchestrator task event rollup`, `AIGuard task event rollup evidence`, `AIGuard runtime operation anomalies` | Shows scheduler delay, deadline misses, fallback decisions, and queue/drop reasons as review context only. |
| Remote starter boundary | `AIGuard remote dispatch event summary`, `Remote fallback starter evidence`, `production_remote_execution=false` | Keeps remote dispatch as starter evidence, not production remote execution. |
Expand Down
1 change: 1 addition & 0 deletions docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ Expected Lab behavior:
- Markdown/HTML reports include a `Runtime Intelligence Risk Summary` that summarizes EdgeEnv comparability/regression, telemetry replay gaps, Runtime history seed/run_config traceability, AIGuard deterministic evidence, and the Lab-owned deployment decision in one reviewer-facing table.
- When EdgeEnv includes preserved Orchestrator feed context, the `Runtime Intelligence Risk Summary` surfaces queue, thermal, throttling, memory, fallback context, and compact `operation_risk_summary` markers as supplemental runtime evidence.
- Lab now keeps queue pressure, `max_total_queue_depth` when available, deadline miss count, and fallback count together in an `Orchestrator queue/deadline/fallback markers` row when those compact counters are present. This is reviewer navigation context, not a production scheduler state or deployment decision override.
- Lab also renders a `Reviewer operation quick scan` row that combines the same queue/deadline/fallback counters with Jetson/device-local preservation identity, so reviewers can identify the relevant EdgeEnv run before opening the full operation rows.
- Lab also surfaces `AIGuard max queue raw-context traceability` when AIGuard
preserves the same `orchestrator_candidate_operation_max_total_queue_depth`
value, making the report row traceable back to deterministic AIGuard raw
Expand Down
53 changes: 53 additions & 0 deletions inferedgelab/report/runtime_intelligence.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,16 @@ def _append_telemetry_context_rows(
)
)

operation_quick_scan_labels = _operation_quick_scan_labels(telemetry_context)
if operation_quick_scan_labels:
rows.append(
(
"Reviewer operation quick scan",
"; ".join(operation_quick_scan_labels),
"One-line navigation labels combine queue/deadline/fallback and Jetson/device-local preservation context without changing Lab policy.",
)
)

operation_risk_labels = _orchestrator_operation_risk_labels(telemetry_context)
if operation_risk_labels:
rows.append(
Expand Down Expand Up @@ -693,6 +703,49 @@ def _orchestrator_queue_deadline_fallback_labels(
return labels


def _operation_quick_scan_labels(context: dict[str, Any]) -> list[str]:
marker_by_run = _label_values_by_run(
_orchestrator_queue_deadline_fallback_labels(context)
)
risk_by_run = _label_values_by_run(_orchestrator_operation_risk_labels(context))
preservation_by_run = _label_values_by_run(
_edgeenv_preservation_run_labels(context)
)
task_by_run = _label_values_by_run(_orchestrator_task_event_rollup_labels(context))

labels: list[str] = []
for run_label in ("baseline", "candidate"):
parts: list[str] = []
marker_label = marker_by_run.get(run_label)
risk_label = risk_by_run.get(run_label)
if marker_label:
parts.append(marker_label)
elif risk_label:
parts.append(f"risk={risk_label}")

preservation_label = preservation_by_run.get(run_label)
if preservation_label:
parts.append(f"preservation={preservation_label}")

if task_by_run.get(run_label):
parts.append("task_rollup=present")

if parts:
labels.append(f"{run_label}: " + "; ".join(parts))
return labels


def _label_values_by_run(labels: list[str]) -> dict[str, str]:
values_by_run: dict[str, str] = {}
for label in labels:
if ": " not in label:
continue
run_label, value = label.split(": ", 1)
if run_label and value and run_label not in values_by_run:
values_by_run[run_label] = value
return values_by_run


def _runtime_replay_scope_labels(context: dict[str, Any]) -> list[str]:
labels: list[str] = []
for run_label in ("baseline", "candidate"):
Expand Down
9 changes: 9 additions & 0 deletions tests/test_report_generators.py
Original file line number Diff line number Diff line change
Expand Up @@ -867,6 +867,13 @@ def test_generate_compare_markdown_summarizes_orchestrator_context_risk():
"queue_pressure_reason=queue_backlog_threshold_exceeded, "
"max_total_queue_depth=7, deadline_missed_count=2, fallback_count=1 |"
) in text
assert (
"| Reviewer operation quick scan | candidate: "
"queue_pressure_reason=queue_backlog_threshold_exceeded, "
"max_total_queue_depth=7, deadline_missed_count=2, fallback_count=1; "
"preservation=identity=jetson_device_local_preservation, "
"run=candidate; task_rollup=present |"
) in text
assert (
"| AIGuard max queue raw-context traceability | candidate: "
"report=max_total_queue_depth=7, "
Expand Down Expand Up @@ -958,10 +965,12 @@ def test_generate_compare_html_summarizes_operation_risk_summary():
assert "scope_label=source=entrypoint_requested_frames" in html
assert "Orchestrator operation risk summary" in html
assert "Orchestrator queue/deadline/fallback markers" in html
assert "Reviewer operation quick scan" in html
assert (
"queue_pressure_reason=queue_backlog_threshold_exceeded, "
"max_total_queue_depth=7, deadline_missed_count=2, fallback_count=1"
) in html
assert "preservation=identity=jetson_device_local_preservation" in html
assert "AIGuard max queue raw-context traceability" in html
assert (
"report=max_total_queue_depth=7, "
Expand Down
9 changes: 9 additions & 0 deletions tests/test_runtime_intelligence_evidence_chain_smoke.py
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,14 @@ def test_compare_cmd_runtime_intelligence_chain_writes_markdown_and_html(
"identity=jetson_device_local_preservation, path=device_local_starter, "
"run=edgeenv-smoke-candidate"
) in markdown
assert (
"| Reviewer operation quick scan | candidate: "
"queue_pressure_reason=queue_backlog_threshold_exceeded, "
"max_total_queue_depth=7, deadline_missed_count=2, fallback_count=1; "
"preservation=identity=jetson_device_local_preservation, "
"path=device_local_starter, run=edgeenv-smoke-candidate; "
"task_rollup=present |"
) in markdown
assert (
"sources=device_local_cli_override, "
"stages=vision_agent:device_local_starter, device_local_events=2, "
Expand Down Expand Up @@ -586,6 +594,7 @@ def test_compare_cmd_runtime_intelligence_chain_writes_markdown_and_html(
assert "Jetson/device-local EdgeEnv preservation run" in html
assert "Jetson/device-local EdgeEnv preservation details" in html
assert "Lab EdgeEnv preservation context" in html
assert "Reviewer operation quick scan" in html
assert "lab_report_preservation_context_present=True" in html
assert "lab_preservation=present" in html
assert "identity=jetson_device_local_preservation" in html
Expand Down
Loading