feat(ui): split assets, expose stats/controls, surface model metadata by igorls · Pull Request #246 · ericcurtin/inferrs

igorls · 2026-05-01T21:31:43Z

Summary

DX: split monolithic ui/index.html (2118 lines) into index.html + src/app.css + src/app.js + vendor/{marked,highlight}.min.js. build.rs walks <link>/<script src> tags and inlines local assets at compile time, so runtime delivery stays a single gzipped blob (currently ~71 KiB, well under the existing 1 MiB budget). cargo:rerun-if-changed walks ui/ recursively.
More UI surface for technical users: stats panel with SERVER · MODEL · RUNTIME · LAST RESPONSE · SESSION sections, per-message TTFT/TPS/tokens footer, live streaming readout, sampling-summary chips above the prompt, color-coded device badge in the navbar (CPU=amber/CUDA=green/Metal=purple), tabbed Settings modal exposing top-k, min-p, repetition/frequency/presence penalty, seed (random/clear), stop sequences, reasoning-effort hint, and behavior toggles.
Better load UX: explicit "Loading model X · Ns" placeholder; polls /api/ps and switches to "Model loaded · generating first token" once the model is actually loaded (so users don't think a slow first token means the load is still running). Cancel-load button + extended global stop button send /api/generate keep_alive=0 so the server unloads instead of finishing the load in the background.
Errors are no longer disguised as Assistant turns: server/network errors now render as a distinct red-bordered system notice and are not pushed into conversation history.
Server-side metadata: new ModelMeta (format / quantization / family / architecture / dtype / device / context_length / vocab_size / shape / turbo_quant_bits / modalities) built at load time from EngineContext. /api/show returns it under inferrs_meta and also populates the Ollama details.{format,family,quantization_level} fields for ecosystem clients. Daemon proxies /api/show to the worker so accurate meta reaches the client.
Misc fix: stripped locally-stored fields (stats) from messages before shipping to /v1/chat/completions — the OpenAI spec only allows {role, content} and strict servers will 400 on extras.

Test plan

Notes

Pre-existing model bugs surfaced during testing (NOT introduced here, listed for visibility):
- ggml-org/gemma-4-26B-A4B-it-GGUF — tensor not found: model.language_model.layers.0.router.proj.weight
- cyankiwi/gemma-4-26B-A4B-it-AWQ-4bit — Failed to parse config.json
- unsloth/Phi-4-mini-instruct-GGUF — missing tokenizer.json on HF (404)
Follow-ups worth considering (out of scope here):
- Populate the quant_method field from HF quantization_config for safetensors-AWQ / GPTQ models (right now we fall back to the model-id heuristic for those)
- Stream usage info on the SSE response (stream_options.include_usage) so the per-message footer can be exact instead of "tokens ≈ chunks"

…e metadata DX - Split monolithic ui/index.html (2118 lines) into ui/index.html (~180 lines of structure) + ui/src/app.css + ui/src/app.js + ui/vendor/{marked,highlight}.min.js. - build.rs walks <link>/<script src> tags and inlines local assets at compile time, so runtime delivery stays a single gzipped blob (1 MiB budget). cargo:rerun-if-changed walks ui/ recursively. UI evolution - Toggleable right stats panel: SERVER (status, endpoint, models loaded, active model), MODEL (format, quantization, family, architecture, ctx, vocab, layers·heads·KV, hidden, modalities), RUNTIME (device, dtype, TurboQuant KV bits), LAST RESPONSE (TTFT, total, tokens, tok/s, finish, reasoning chunks), SESSION counters. - Per-message stats footer on each assistant turn (TTFT · tokens · tok/s · total). - Live readout at the bottom of the input area while streaming. - Sampling-summary chips above the prompt showing only the active overrides. - Tabbed Settings modal (Basic / Sampler / Behavior) exposing top-k, min-p, repetition / frequency / presence penalty, seed (random/clear), stop sequences, reasoning effort, plus toggles for stats panel default, reasoning auto-expand, and stream-render markdown. - Color-coded device badge in the navbar (CPU=amber warning, CUDA=green, Metal=purple) so users immediately see which path they're on. - Server status pill polls /health every 5s. - Loading-state placeholder with elapsed seconds; polls /api/ps to switch from "Loading model X" to "Model loaded · generating first token" once the model is actually ready. - Cancel-load button (also extends the global stop button) — sends /api/generate keep_alive=0 so the server doesn't keep loading in the background after the user changes their mind. - Errors now render as a distinct red-bordered system notice rather than being styled as fake "Assistant" turns and saved to history. - Per-conversation delete + "Clear all history" + Reset to defaults. - Strip locally-stored fields (`stats`) from messages before shipping — OpenAI spec only allows {role, content}. Server-side metadata exposure - New ModelMeta struct (format, quantization, family, architecture, dtype, device, context_length, vocab_size, layer/head shape, turbo_quant_bits, has_audio/vision) built at load time from EngineContext. - /api/show now returns inferrs_meta + populates Ollama details.format, details.family, details.quantization_level. Daemon mode proxies the request to the worker so accurate meta reaches the client. - Quantization heuristic from model id covers Q4_K_M, Q8_0, AWQ-4bit, GPTQ-int4, FP16, BF16, FP8, etc. without pulling in a regex crate.

Copilot

Pull request overview

This PR refactors the built-in web UI into split assets that are bundled/inlined at compile time, expands the UI to expose runtime/model stats and advanced sampling controls, and adds server-side model metadata surfaced via /api/show for the UI and Ollama-compat clients.

Changes:

Split UI into ui/index.html + ui/src/{app.css,app.js} + vendored marked/highlight.js, with build.rs inlining local assets into a single gzipped HTML blob at build time.
Add a richer UI: stats panel (server/model/runtime/last response/session), device badge, streaming status/readout, settings modal for advanced sampling and behavior toggles, and clearer error notices.
Add server-side ModelMeta captured at load time and returned from /api/show (and proxy /api/show to the worker in daemon mode); extend EngineContext with fields needed to build this snapshot.

Reviewed changes

Copilot reviewed 5 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`inferrs/ui/vendor/marked.min.js`	Vendor markdown parser added for UI markdown rendering.
`inferrs/ui/vendor/highlight.min.js`	Vendor highlight.js added to support code highlighting in UI.
`inferrs/ui/src/app.js`	New main UI logic: chat, streaming, settings, stats, metadata fetching, error handling.
`inferrs/ui/src/app.css`	New UI styling including stats panel, settings modal, device badge, and error notice styles.
`inferrs/src/server.rs`	Add `ModelMeta`, include it in `/api/show`, and proxy `/api/show` to worker in daemon mode.
`inferrs/src/engine.rs`	Extend `EngineContext` with device/quant/gguf/turbo-quant fields for metadata reporting.
`inferrs/build.rs`	Implement UI bundling by inlining local `<link>`/`<script src>` assets then gzipping.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

gemini-code-assist

Code Review

This pull request implements a Web UI bundling system in the build script and introduces a new browser interface for model interaction. It also updates the engine and server to capture and report detailed model metadata, such as architecture and device information, via the /api/show endpoint. The review feedback highlights critical security issues, including a path traversal vulnerability in the build script and an XSS risk in the Web UI's markdown rendering.

gemini-code-assist · 2026-05-01T21:36:05Z

+            // Leave external references untouched.
+            out.push_str(&html[abs_start..after_tag]);
+        } else {
+            let path = PathBuf::from(UI_DIR).join(url);


The asset inlining logic is vulnerable to path traversal. The url extracted from the HTML is joined with the ui/ directory without validation. A malicious index.html could use relative paths (e.g., ../../etc/passwd) or absolute paths to read and inline sensitive files from the build environment into the final binary. This constitutes a supply chain risk as defined in the repository style guide.

Suggested change

let path = PathBuf::from(UI_DIR).join(url);

let url_path = std::path::Path::new(url);

if url_path.is_absolute() || url_path.components().any(|c| matches!(c, std::path::Component::ParentDir)) {

panic!("malicious path in UI asset: {}", url);

}

let path = PathBuf::from(UI_DIR).join(url);

References

Security — What are the threat surfaces? ... Is input validated at system boundaries? ... Think about the OWASP top 10, supply chain risks... ^(link)

gemini-code-assist · 2026-05-01T21:36:05Z

+      + thinking.length + ' block' + (thinking.length > 1 ? 's' : '') + ')</summary><div>'
+      + escHtml(thinking.join('\n\n')) + '</div></details>';
+  }
+  html += marked.parse(cleaned);


The assistant's output is rendered as HTML using marked.parse() and injected into the DOM via innerHTML without any sanitization. This allows a malicious or compromised model to execute arbitrary JavaScript in the user's browser (XSS). Since this UI is intended for technical users who may interact with untrusted models, this is a significant security risk. Please use a sanitization library like DOMPurify to clean the HTML before injection.

References

Security — What are the threat surfaces? ... Think about the OWASP top 10... ^(link)

greptile-apps · 2026-05-01T21:36:51Z

Confidence Score: 3/5

Safe to merge after addressing the absolute-path guard in is_external and evaluating the XSS risk from unsanitized markdown rendering.

One P1 (absolute-path bypass in build script poses a supply-chain integrity risk) and two P2s (attribute-order brittleness in the bundler; unsanitized marked.parse innerHTML) pull the score below 4.

inferrs/build.rs (path traversal in is_external) and inferrs/ui/src/app.js (XSS from unsanitized markdown output).

Security Review

XSS via unsanitized marked.parse output (ui/src/app.js line 87): LLM-generated HTML is set as innerHTML without a sanitization pass; raw <script> or event-handler attributes in a model response would execute in the browser.
Build-time path traversal in build.rs (line 151–156): is_external does not block absolute paths (e.g. /etc/shadow); PathBuf::join with an absolute component replaces the accumulated path, allowing arbitrary host files to be silently embedded in the binary during a CI build if ui/index.html is modified.

Important Files Changed

Filename	Overview
inferrs/build.rs	New HTML-bundling logic inlines CSS/JS at build time; attribute-order rigidity and missing absolute-path guard in `is_external` are two logic issues.
inferrs/src/server.rs	Adds `ModelMeta` struct, `build_model_meta`, daemon proxy for `/api/show`, and populates Ollama `details` fields; logic is sound, proxy and serde skip conditions behave correctly.
inferrs/src/engine.rs	Adds `device`, `quant_dtype`, `gguf_loaded`, and `turbo_quant_bits` to `EngineContext`; straightforward metadata capture with no logic issues.
inferrs/ui/src/app.js	New 1073-line UI script; overall well-structured, but `marked.parse` output is set as `innerHTML` without sanitization (XSS risk from LLM output), and minor null-contract issue in `computeStats`.
inferrs/ui/index.html	Monolithic HTML split into skeleton with `<link>`/`<script>` references that build.rs inlines; structure is correct for the bundling approach.
inferrs/ui/src/app.css	New CSS file extracted from the monolithic HTML; new styles for stats panel, device badge, sampling chips, and error notices look correct.

_{Reviews (1): Last reviewed commit: "feat(ui): split assets, expose stats/con..." | Re-trigger Greptile}

greptile-apps · 2026-05-01T21:36:56Z

+fn is_external(url: &str) -> bool {
+    url.starts_with("http://")
+        || url.starts_with("https://")
+        || url.starts_with("//")
+        || url.starts_with("data:")
 }


Absolute paths bypass is_external guard

is_external only rejects http://, https://, //, and data: URLs. An absolute path like /etc/shadow passes through, and PathBuf::from("ui").join("/etc/shadow") resolves to /etc/shadow (Rust's PathBuf::join replaces the accumulated path when the new component is absolute). If ui/index.html ever references an absolute href/src, the build script would embed arbitrary host files into the binary at compile time — a silent supply-chain risk in CI.

Add an absolute-path check:

Suggested change

fn is_external(url: &str) -> bool {

url.starts_with("http://")

|| url.starts_with("https://")

|| url.starts_with("//")

|| url.starts_with("data:")

}

fn is_external(url: &str) -> bool {

url.starts_with("http://")

|| url.starts_with("https://")

|| url.starts_with("//")

|| url.starts_with("data:")

|| url.starts_with('/')

}

greptile-apps · 2026-05-01T21:36:57Z

+    let html = inline_tag(
+        html,
+        // <link rel="stylesheet" href="..."> — order-tolerant: rel before href
+        r#"<link rel="stylesheet" href=""#,
+        r#"">"#,
+        |body| format!("<style>\n{body}\n</style>"),
    );
+    inline_tag(
+        &html,
+        r#"<script src=""#,
+        r#"></script>"#,
+        |body| format!("<script>\n{body}\n</script>"),
+    )


Strict attribute-order assumption in link-tag matching

inline_tag searches for the literal prefix <link rel="stylesheet" href=", so it only inlines CSS when rel comes before href. If a developer writes <link href="src/app.css" rel="stylesheet"> (or a tool reorders the attributes), the <link> tag is left verbatim in the bundle. The browser then tries to GET /src/app.css which the server doesn't serve — the page renders without styles, and the build succeeds at the 1 MiB budget check with no warning.

Consider matching on href alone (or at least asserting a <link rel="stylesheet" is present and inlined at the end of the build step).

greptile-apps · 2026-05-01T21:36:58Z

+  html += marked.parse(cleaned);
+  return html;


LLM-generated HTML rendered without sanitization

marked.parse(cleaned) is piped directly into assistantBody.innerHTML. Because marked v4+ no longer auto-strips raw HTML (the old sanitize option was removed), a model response containing <script>alert(1)</script> or <img src=x onerror=...> would execute in the browser context. Although the immediate threat model is local-only, a jailbroken or model-poisoned response could exfiltrate localStorage (which holds conversation history and settings) or trigger unexpected side-effects.

Consider adding a lightweight sanitizer such as DOMPurify or replacing arbitrary HTML nodes with a safe subset of tags via marked's walkTokens / renderer hooks.

…ibute order, fmt Security - build.rs: reject absolute paths and `..` components when resolving asset URLs to be inlined. An absolute `href`/`src` made `PathBuf::from("ui"). join("/etc/shadow")` resolve to `/etc/shadow` and silently embed host files into the binary at compile time. Treat such URLs as a hard build failure. Strip `?` / `#` from URLs before resolving. Flagged P1 by both greptile and gemini bots. - ui/src/app.js: marked v15 dropped its `sanitize` option, so raw HTML in model output (`<script>`, `onerror=`, `javascript:` URLs) was reaching `innerHTML`. Override `marked.Renderer` to (1) escape `html` tokens rather than emit them verbatim, and (2) drop `javascript:` / `data:` / `vbscript:` URLs in links and images. Verified in-browser with a payload containing `<script>`, `<img onerror>`, and a `javascript:` link — none execute, legitimate markdown still renders. Correctness - build.rs: tag matcher is now tolerant of attribute order — previously `<link href="x" rel="stylesheet">` (href before rel) would not be inlined, the daemon doesn't serve those paths, and the UI shipped unstyled. Walks the tag, parses attributes individually, requires `rel="stylesheet"` for `<link>` and `src=…` for `<script>`. - build.rs: belt-and-suspenders post-bundle assertion fails the build if any local `<link href>` / `<script src>` survives the bundling pass. Lint - Apply `cargo fmt` (CI was failing on `cargo fmt --check`).

igorls · 2026-05-01T21:48:56Z

Addressed all review feedback in fbb7659:

Security

Path traversal in build.rs (Greptile P1, Gemini critical): URLs are now run through sanitize_local_url which rejects absolute paths and .. components, plus is_external treats absolute paths (starting with /) as external so they're left in place rather than resolved against ui/. Query strings and fragments are stripped before path resolution.
XSS via marked.parse (Greptile P2 / Gemini critical): added a safeRenderer that escapes raw HTML tokens (<script>, <img onerror>, etc.) and drops javascript: / data: / vbscript: URLs in links and images. Verified in-browser with a payload containing all three vectors — none execute, window.__pwned* stay undefined, document.title unchanged, but legitimate **bold** and `code` still render. Chosen over vendoring DOMPurify (~24 KiB) since the override is ~20 lines and covers the realistic LLM-output XSS surface.

Correctness

Attribute-order brittleness (Greptile P2): rewrote the tag matcher to walk <link> / <script> opening tags, parse attributes individually, and require rel="stylesheet" (any token order) and src=… regardless of attribute order. <link href="x" rel="stylesheet"> now bundles correctly.
Belt-and-suspenders assertion: after bundling, the build fails if any <link href> / <script src> for a local URL survives — catches future parser misses at compile time instead of shipping a silently-broken UI.

Lint

Ran cargo fmt -p inferrs; cargo fmt --check -p inferrs is now clean.

Copilot AI review requested due to automatic review settings May 1, 2026 21:31

Copilot started reviewing on behalf of igorls May 1, 2026 21:32 View session

Copilot AI reviewed May 1, 2026

View reviewed changes

gemini-code-assist Bot reviewed May 1, 2026

View reviewed changes

greptile-apps Bot reviewed May 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ui): split assets, expose stats/controls, surface model metadata#246

feat(ui): split assets, expose stats/controls, surface model metadata#246
igorls wants to merge 2 commits into
ericcurtin:mainfrom
igorls:feat/web-ui-evolution

igorls commented May 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 1, 2026

Uh oh!

gemini-code-assist Bot May 1, 2026

Uh oh!

greptile-apps Bot commented May 1, 2026

Security Review

Uh oh!

greptile-apps Bot May 1, 2026

Uh oh!

greptile-apps Bot May 1, 2026

Uh oh!

greptile-apps Bot May 1, 2026

Uh oh!

igorls commented May 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-            let path = PathBuf::from(UI_DIR).join(url);
+            let url_path = std::path::Path::new(url);
+            if url_path.is_absolute() || url_path.components().any(|c| matches!(c, std::path::Component::ParentDir)) {
+                panic!("malicious path in UI asset: {}", url);
+            }
+            let path = PathBuf::from(UI_DIR).join(url);

Conversation

igorls commented May 1, 2026

Summary

Test plan

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 1, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented May 1, 2026

Confidence Score: 3/5

Security Review

Important Files Changed

Uh oh!

greptile-apps Bot May 1, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 1, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 1, 2026

Choose a reason for hiding this comment

Uh oh!

igorls commented May 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants