feat(ui): live progress + speed for library/component downloads (#128)

Library / component dependency downloads now show the same live progress
bar, byte counts and transfer speed that toolchain downloads already show.

Three paths fed download events; only toolchain and builtin-index installs
rendered progress. Project / custom-index dependency installs went through
the silenced direct xlings CLI and ran dark — "Downloading <pkg>" then a
long, feedback-free hang.

- Route project/custom-index installs through the xlings NDJSON
  `interface install_packages` capability (the capability and the `install`
  CLI share `cmd_install`; install destination is chosen by package scope,
  so packages still land in the project-local data root). This restores the
  live bar on that path.
- Add an indeterminate ("connecting…") render for the pre-sizing window
  where the downloader reports totalBytes==0 (DNS/TLS/redirect, or a body
  streamed with no Content-Length), so the line never freezes.
- Centralize the download-progress state machine + render policy in
  `mcpp.ui` (DownloadProgress); toolchain, builtin-index and custom-index
  installs now share one UI.
- Pin bundled xlings to 0.4.51.

Bumps version to 0.0.53. Updates e2e 52/58/60 to expect the interface
transport while keeping their project-local-install and hook-ordering
invariants. Design: .agents/docs/2026-06-09-library-download-progress-design.md

Author: Sunrisepeak

.agents/docs/2026-06-09-library-download-progress-design.md

@@ -0,0 +1,162 @@
+# Library / Component Download Progress — Design
+
+> Status: approved, in implementation
+> Target release: mcpp 0.0.53
+> Scope: make library / component downloads show the same live progress + speed
+> that toolchain downloads already show, across Linux / macOS / Windows.
+
+## 1. Problem
+
+`mcpp toolchain install` shows a live progress bar with percent, bytes and
+transfer speed. Installing **library / component dependencies** (during
+`mcpp build`, dependency resolution, or `mcpp new --template`) prints a single
+`Downloading <pkg> v<ver>` line and then **appears to hang** for a long time with
+no percent, no speed and no sign of life — especially for large packages on slow
+mirrors.
+
+## 2. Root cause (evidence-based)
+
+There are **three** download code paths. They were not equally instrumented.
+
+| Path | Trigger | Transport | Symptom |
+|------|---------|-----------|---------|
+| ① Toolchain | `mcpp toolchain install` | NDJSON `interface install_packages` + `CliInstallProgress` | progress OK |
+| ② Builtin-index deps | dep resolves via builtin index (`useProjectEnv = false`) | same as ① | progress OK *once bytes flow* |
+| ③ Custom/project-index deps | dep resolves via a project-added index (`useProjectEnv = true`) | **`install_direct(projEnv, target, quiet=true)`** | **fully silent** |
+
+Two independent defects fall out of this:
+
+### Defect A — path ③ is silenced
+
+`src/cli.cppm` routes project/custom-index dependency installs through
+`mcpp::xlings::install_direct(projEnv, target, /*quiet=*/true)`. `quiet=true`
+redirects xlings' stdout/stderr to the platform null device, so xlings' own
+progress output is discarded and **no NDJSON events are parsed** (the direct CLI
+does not speak NDJSON). The `Downloading <pkg>` line is printed by mcpp *before*
+this call, then the whole download runs dark.
+
+This path was switched to the direct CLI in a prior change to guarantee that
+project-index packages land in the **project-local** xlings data root (so a
+package's install hook can find sibling packages from the same index). That
+change traded away progress as a side effect.
+
+**Why the NDJSON interface is in fact safe here (verified against xlings
+source):** in the pinned xlings, the `install_packages` *capability* and the
+`install` *CLI* both call the same `xim::cmd_install(...)`. The install
+destination is chosen by **package scope** — `storeRoot = (scope == Project ?
+project_data_dir() : global_data_dir()) / "xpkgs"` — which is derived from
+*which index the package came from*, not from interface-vs-CLI. `download_progress`
+events are emitted unconditionally (no scope gate). So the NDJSON interface
+installs project-scoped packages into the project-local data root **and** streams
+progress. The earlier switch was working around index *exposure*, which is now
+handled separately (the project index is symlinked/exposed into the project data
+dir and targets are passed as `indexName:fqname@version`).
+
+### Defect B — the "connecting" phase freezes the line (all NDJSON paths)
+
+Captured NDJSON for a real package download shows the downloader emits
+`download_progress` events with `totalBytes = 0` (and often `downloadedBytes = 0`)
+during the initial connect / TLS / redirect / pre-sizing window, before the
+transfer's total size is known:
+
+```
+elapsed 0.2s–1.4s : downloadedBytes=0  totalBytes=0  sizesReady=false   (connecting)
+elapsed 1.6s      : downloadedBytes=57344    totalBytes=1754511         (bytes flow)
+elapsed 2.0s      : downloadedBytes=1754511  totalBytes=1754511  finished
+```
+
+Both `CliInstallProgress::on_data` and `make_bootstrap_progress_callback` only
+call `ProgressBar::update_bytes` when `total > 0`, so during the connecting
+window the line shows nothing. For a large file on a slow mirror this window can
+last many seconds and reads as a hang. Toolchain downloads hide this because the
+connect window is tiny relative to a multi-minute transfer; library downloads do
+not.
+
+## 3. Design
+
+Three changes, all behind existing abstractions, all cross-platform.
+
+### 3.1 Fix 1 — path ③ uses the NDJSON interface (mcpp-style bar)
+
+In `src/cli.cppm`, the `useProjectEnv` branch of the dependency install lambda
+calls the NDJSON interface with the project env and an `CliInstallProgress`
+handler, instead of the silenced direct CLI:
+
+```cpp
+if (useProjectEnv) {
+    auto projEnv  = mcpp::config::make_project_xlings_env(**cfg, *root);
+    auto argsJson = std::format(R"({{"targets":["{}"],"yes":true}})", target);
+    CliInstallProgress progress;
+    auto r = mcpp::xlings::call(projEnv, "install_packages", argsJson, &progress);
+    if (!r) return std::unexpected(mcpp::pm::CallError{r.error()});
+    return *r;
+}
+```
+
+This yields the same cyan `Downloading … [bar] X/Y  Z/s` UI on **all** three
+paths. Package scope (and therefore install location) is unchanged — packages
+from a project index still install into the project-local data root.
+
+### 3.2 Fix 2 — indeterminate rendering while size is unknown
+
+Add `ProgressBar::update_indeterminate(current_bytes, elapsed_sec)`: render a
+swept/indeterminate bar plus an info suffix that ticks — `connecting…  Ns`, or a
+byte counter `X.Y MB  Ns` once bytes arrive without a known total. This also
+covers servers that stream without `Content-Length`.
+
+Wire both progress consumers (`CliInstallProgress::on_data` and
+`make_bootstrap_progress_callback`) to call `update_indeterminate` when the
+active file has `total == 0` (and `update_bytes` once `total > 0`).
+
+Refactor `render_line` so the bar string is produced by a small callable; the
+percent path and the indeterminate path share the terminal-width budgeting.
+
+### 3.3 Fix 3 — pin xlings to the latest release
+
+Bump `pinned::kXlingsVersion` in `src/xlings.cppm` to the latest xlings release
+(the version that ships the unified `cmd_install` + scope-routed install +
+unconditional `download_progress`). This removes any dependence on older xlings
+behavior for the path ③ install location.
+
+## 4. Cross-platform
+
+- Progress rendering already runs on Linux/macOS/Windows (toolchain path uses it
+  today); no new platform branches are introduced.
+- `mcpp::xlings::call` / `build_interface_command` already handle the Windows vs
+  POSIX command shape and env propagation; path ③ inherits that.
+- `update_indeterminate` uses the same `render_line` budgeting (TIOCGWINSZ /
+  `$COLUMNS` / 80-col fallback) as the existing bar; non-TTY / `--quiet` paths
+  remain silent via the existing `g_quiet` guard.
+
+## 5. Testing
+
+- **Unit-ish / build:** project builds clean with the existing warning gate.
+- **E2E regression:** `tests/e2e/52_local_path_namespaced_index.sh` and
+  `tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh` currently assert path ③ uses
+  the *direct* CLI. They are updated to assert the **interface** transport while
+  keeping their existing invariants (project-local install location, and
+  `mcpp.deps` installed before a dependent package's install hook runs).
+- **Real integration check:** with a real project-local custom index and a real
+  installable package, confirm (a) the package installs into the project-local
+  data root, (b) the install hook still sees its `mcpp.deps`, and (c) the
+  `Downloading … [bar] X/Y Z/s` UI renders. This is the empirical gate before
+  merge.
+
+## 6. Rollout
+
+1. Implement Fix 1–3; update e2e tests.
+2. Local build + e2e + real integration check.
+3. Bump `MCPP_VERSION` (`src/toolchain/fingerprint.cppm`) and `mcpp.toml` to
+   `0.0.53`; CHANGELOG entry.
+4. PR → CI green → squash merge → trigger release `0.0.53`.
+5. Ecosystem: add mcpp `0.0.53` to the package index (mcpp xpkg descriptor:
+   version + per-platform sha256), publish release artifacts to the
+   release-asset mirrors, PR + merge, verify an end-to-end install of the new
+   version.
+
+## 7. Out of scope / follow-ups
+
+- No change to xlings' own native CLI progress UI.
+- No change to the dependency-resolution / preinstall ordering logic (it already
+  installs deps before dependents; only the per-install transport on path ③
+  changes).

.xlings.json

@@ -1,5 +1,5 @@
 {
   "workspace": {
-    "mcpp": "0.0.52"
+    "mcpp": "0.0.53"
   }
 }

CHANGELOG.md

@@ -3,6 +3,26 @@
 > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。
 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。
 
+## [0.0.53] — 2026-06-09
+
+### 新增
+
+- 库 / 组件下载现在与工具链下载一样显示实时进度条、字节进度与速度。自定义 /
+  项目索引依赖改经 xlings NDJSON `interface install_packages` 安装(仍落在项目
+  本地数据根,不改变安装位置与 install hook 顺序),不再静默卡住。
+
+### 修复
+
+- 下载连接 / 预取大小阶段(`totalBytes` 尚未知)进度行不再"冻结"无反馈:
+  新增不确定态渲染,显示 `connecting…` + 已用时,流式无 `Content-Length`
+  时显示已下载字节,直到拿到总大小再切换为百分比进度条。
+
+### 其他
+
+- 内置 xlings 版本上调至 `0.4.51`。
+- 下载进度的状态机与渲染集中到 `mcpp.ui`(`DownloadProgress`),工具链 /
+  内置索引 / 自定义索引三条路径共用同一套 UI。
+
 ## [0.0.46] — 2026-06-03
 
 ### 新增

mcpp.toml

@@ -1,6 +1,6 @@
 [package]
 name        = "mcpp"
-version     = "0.0.52"
+version     = "0.0.53"
 description = "Modern C++ build & package management tool"
 license     = "Apache-2.0"
 authors     = ["mcpp-community"]

src/cli.cppm

@@ -448,119 +448,52 @@ mcpp::ui::PathContext make_path_ctx(const mcpp::config::GlobalConfig* cfg,
     return ctx;
 }
 
-// Stateless adapter from `mcpp::config::BootstrapProgress` (xlings
-// download_progress event) to a sticky ProgressBar. Used by
-// load_or_init() during the one-time sandbox bootstrap (xim:patchelf,
-// xim:ninja, plus their transitive deps).
-//
-// Two xlings quirks the callback has to absorb:
-//   1. Each file's `finished=true` event arrives twice in a row.
-//   2. During multi-package installs the `files[]` array reshuffles
-//      between events (the active download isn't always at slot 0).
-// The fix mirrors CliInstallProgress: dedupe via a `finished_` set and
-// always pick "active first if still in event, else first
-// started+unfinished" rather than reading slot 0 blindly.
-mcpp::config::BootstrapProgressCallback make_bootstrap_progress_callback() {
-    auto bar      = std::make_shared<std::optional<mcpp::ui::ProgressBar>>();
-    auto active   = std::make_shared<std::string>();
-    auto finished = std::make_shared<std::unordered_set<std::string>>();
-    return [bar, active, finished](const mcpp::config::BootstrapProgress& ev) {
-        // Process newly-finished entries.
-        for (auto& f : ev.files) {
-            if (finished->contains(f.name)) continue;
-            if (!f.finished) continue;
-            if (*active == f.name) {
-                if (*bar) (*bar)->finish();
-                bar->reset();
-                active->clear();
-            }
-            finished->insert(f.name);
-        }
-
-        // Pick what to display: prefer continuing with `*active` if it's
-        // still in the array and not finished, otherwise the first
-        // started+unfinished entry.
-        const mcpp::config::BootstrapFile* current = nullptr;
-        for (auto& f : ev.files) {
-            if (f.name == *active && !f.finished
-                && !finished->contains(f.name)) { current = &f; break; }
-        }
-        if (!current) {
-            for (auto& f : ev.files) {
-                if (finished->contains(f.name)) continue;
-                if (f.started && !f.finished) { current = &f; break; }
-            }
+// Map a decoded NDJSON `download_progress` files[] snapshot onto the neutral
+// `mcpp::ui::DownloadFile` the centralized renderer consumes.
+template <class File>
+std::vector<mcpp::ui::DownloadFile> to_ui_download_files(const std::vector<File>& files) {
+    std::vector<mcpp::ui::DownloadFile> out;
+    out.reserve(files.size());
+    for (auto& f : files) {
+        if constexpr (requires { f.downloadedBytes; }) {
+            out.push_back({ f.name,
+                            static_cast<std::size_t>(f.downloadedBytes),
+                            static_cast<std::size_t>(f.totalBytes),
+                            f.started, f.finished });
+        } else {
+            out.push_back({ f.name,
+                            static_cast<std::size_t>(f.downloaded),
+                            static_cast<std::size_t>(f.total),
+                            f.started, f.finished });
         }
-        if (!current) return;
+    }
+    return out;
+}
 
-        if (current->name != *active) {
-            if (*bar) (*bar)->finish();
-            *active = current->name;
-            bar->emplace("Downloading", current->name);
-        }
-        if (current->totalBytes > 0) {
-            (*bar)->update_bytes(static_cast<std::size_t>(current->downloadedBytes),
-                                 static_cast<std::size_t>(current->totalBytes),
-                                 ev.elapsedSec);
-        }
+// Adapter from `mcpp::config::BootstrapProgress` (xlings download_progress
+// event) to the centralized download renderer. Used by load_or_init() during
+// the one-time sandbox bootstrap (xim:patchelf, xim:ninja + transitive deps).
+mcpp::config::BootstrapProgressCallback make_bootstrap_progress_callback() {
+    auto progress = std::make_shared<mcpp::ui::DownloadProgress>();
+    return [progress](const mcpp::config::BootstrapProgress& ev) {
+        auto files = to_ui_download_files(ev.files);
+        progress->update(files, ev.elapsedSec);
     };
 }
 
+// EventHandler that forwards xlings `download_progress` events to the same
+// centralized renderer. Used for toolchain, builtin-index and custom-index
+// installs alike, so all three show identical UI.
 struct CliInstallProgress : mcpp::fetcher::EventHandler {
-    std::optional<mcpp::ui::ProgressBar> bar_;
-    std::string                          active_;
-    std::unordered_set<std::string>      finished_;
+    mcpp::ui::DownloadProgress progress_;
 
     void on_data(const mcpp::fetcher::DataEvent& d) override {
         if (d.dataKind != "download_progress") return;
         auto files = parse_all_install_files(d.payloadJson);
         if (files.empty()) return;
-
-        // 1. Process any newly-finished entries. Each file is reported
-        //    twice with finished=true (xlings quirk); the `finished_`
-        //    set dedupes both that AND the rotation case where the
-        //    same file shows up at a different array slot in a later
-        //    event.
-        for (auto& f : files) {
-            if (finished_.contains(f.name)) continue;
-            if (!f.finished) continue;
-            if (active_ == f.name) {
-                if (bar_) bar_->finish();
-                bar_.reset();
-                active_.clear();
-            }
-            finished_.insert(f.name);
-        }
-
-        // 2. Pick what to display. Prefer continuing with the current
-        //    `active_` if it's still in the array and not finished —
-        //    otherwise the first started+unfinished entry. This stops
-        //    the bar from flickering between names when xlings reshuffles
-        //    files[] across events during a multi-package install.
-        const InstallProgressFile* current = nullptr;
-        for (auto& f : files) {
-            if (f.name == active_ && !f.finished
-                && !finished_.contains(f.name)) { current = &f; break; }
-        }
-        if (!current) {
-            for (auto& f : files) {
-                if (finished_.contains(f.name)) continue;
-                if (f.started && !f.finished) { current = &f; break; }
-            }
-        }
-        if (!current) return;
-
-        if (current->name != active_) {
-            if (bar_) bar_->finish();
-            active_ = current->name;
-            bar_.emplace("Downloading", current->name);
-        }
-        if (current->total > 0) {
-            double elapsed = extract_payload_number(d.payloadJson, "elapsedSec");
-            bar_->update_bytes(static_cast<std::size_t>(current->downloaded),
-                               static_cast<std::size_t>(current->total),
-                               elapsed);
-        }
+        double elapsed = extract_payload_number(d.payloadJson, "elapsedSec");
+        auto ui_files = to_ui_download_files(files);
+        progress_.update(ui_files, elapsed);
     }
 
     void on_log(const mcpp::fetcher::LogEvent& e) override {
@@ -579,7 +512,8 @@ struct CliInstallProgress : mcpp::fetcher::EventHandler {
             mcpp::log::info("xlings", std::format("hint: {}", e.hint));
     }
 
-    ~CliInstallProgress() override { if (bar_) bar_->finish(); }
+    // progress_'s own destructor finishes the active bar.
+    ~CliInstallProgress() override = default;
 };
 
 // Compose a stable canonical compile-flags string for fingerprinting.
@@ -2312,11 +2246,24 @@ prepare_build(bool print_fingerprint,
 
             auto install_one = [&](std::string target) -> std::expected<mcpp::xlings::CallResult, mcpp::pm::CallError> {
                 if (useProjectEnv) {
+                    // Project/custom-index deps install into the project-local
+                    // xlings data root (so a package's install hook can find
+                    // sibling packages from the same index). The NDJSON
+                    // interface honors this: in the pinned xlings the
+                    // `install_packages` capability and the `install` CLI share
+                    // `xim::cmd_install`, and the install destination is chosen
+                    // by package *scope* (project vs global), not by transport.
+                    // Using the interface (rather than the silenced direct CLI)
+                    // restores the live `Downloading … [bar] X/Y Z/s` UI here,
+                    // matching the toolchain and builtin-index paths.
                     auto projEnv = mcpp::config::make_project_xlings_env(**cfg, *root);
-                    int directRc = mcpp::xlings::install_direct(projEnv, target, /*quiet=*/true);
-                    mcpp::xlings::CallResult result;
-                    result.exitCode = directRc;
-                    return result;
+                    auto argsJson = std::format(
+                        R"({{"targets":["{}"],"yes":true}})", target);
+                    CliInstallProgress progress;
+                    auto r = mcpp::xlings::call(
+                        projEnv, "install_packages", argsJson, &progress);
+                    if (!r) return std::unexpected(mcpp::pm::CallError{r.error()});
+                    return *r;
                 }
                 std::vector<std::string> targets{ std::move(target) };
                 CliInstallProgress progress;