feat: graph statistics panel with density, components, reciprocity, degree histogram#424
Closed
shaunpatterson wants to merge 55 commits into
Closed
feat: graph statistics panel with density, components, reciprocity, degree histogram#424shaunpatterson wants to merge 55 commits into
shaunpatterson wants to merge 55 commits into
Conversation
Every component test suite has been failing to run. Four independent breakages, all in test infrastructure - no production code changes: - ESM-only packages were never transpiled: the babel config lives in package.json (file-relative, like .babelrc), so it is not applied to files inside node_modules even when transformIgnorePatterns allows them through. Added config/jest/babelTransform.js (explicit presets) and allowed react-leaflet/@react-leaflet through the transform. - Jest 26 cannot resolve node:-prefixed core modules required by newer transitive deps (cheerio -> parse5/undici). Added shim files under config/jest/nodeShims plus a moduleNameMapper rule. - enzyme 3 requires cheerio/lib/utils, which no longer exists in cheerio 1.x final (the lockfile resolves enzyme's ^1.0.0-rc.3 range to 1.1.2). Mapped to its new location (dist/commonjs/utils.js). - jsdom 16 lacks TextEncoder/TextDecoder, web streams, Blob and MessageChannel globals that undici needs. Added a Jest-only setupFiles polyfill (kept out of config/polyfills.js, which is also a webpack entry). Also split e2e tests out of the default run: they need puppeteer and a live Dgraph cluster, so 14 suites always failed in a plain checkout. 'npm test' now runs unit tests only; 'npm run test:e2e' runs the rest. Result: npm test goes from 16/18 suites failing to 4/4 passing (13 tests). Production build verified unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds two buttons to the graph toolbar (next to zoom-to-fit): - PNG: composites every canvas in the graph container onto a single white-backed canvas and downloads it via toBlob, so it works with the current d3 canvas renderer and any future layered renderer. - JSON: serializes the GraphParser node/edge Maps to plain JSON. Edge endpoints may be uid strings or node objects resolved in place by d3-force - both shapes export as uids, keeping the output free of circular references. lib/exportGraph.js is covered by 8 unit tests. Verified in a real browser against a live Dgraph cluster: both buttons download valid files (the PNG renders the graph, the JSON carries correct uid endpoints). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap the hand-rolled d3-force canvas renderer (~1100 lines of manual hit-testing, arc math and label culling) for sigma.js v3 + graphology: - WebGL rendering scales to tens of thousands of nodes (the canvas renderer struggles past a few hundred); labels get automatic collision/density handling. - ForceAtlas2 layout runs in a web worker, so the UI thread never blocks during layout; it auto-stops after 4s. - Parallel edges between the same node pair fan out as distinct curves (@sigma/edge-curve), preserving the sibling-edge behavior. - Nodes sized by degree (capped); neighbor highlighting on hover; drag-to-pin; double-click to expand/collapse preserved. - Implements the GraphContainer ref API (searchNode/focusNode/ zoomToFit) with the same matching semantics as the d3 renderer. - buildGraph keeps the d3-force contract of resolving edge source/target uids to node objects in place - EdgeProperties and GraphParser.collapseNode depend on that mutation. Covered by 12 unit tests. - sigma is mocked under Jest (jsdom has no WebGL2RenderingContext). - scripts/graph-smoke.mjs: puppeteer smoke test that seeds a local Dgraph, logs in, runs a query through the real UI and asserts the WebGL canvases render and search/zoomToFit work. - Drop the d3 dependency (D3Graph was its only consumer). Verified: 12/12 buildGraph tests, production build, and the browser smoke test against Dgraph v25 with ACL (3 nodes / 3 edges rendered via WebGL, search + zoom-to-fit exercised, zero page errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dgraph returns a per-phase latency breakdown with every response (extensions.server_latency: parsing/processing/encoding/...), but Ratel discarded it - and the frame header latency bar was dead code: it read frame.serverLatencyNs while the timing lives on frameResults[id][tab], so it never rendered at all. - lib/latency.js: pure helpers turning server_latency into ordered, labelled bar segments (known phases in pipeline order, unknown *_ns fields included future-proof, total_ns excluded) plus tooltip text. 9 unit tests. - frames reducer keeps the raw server_latency on the frame result. - FrameHeader now receives tabResult and renders a multi-segment color-coded bar (parsing/processing/encoding/network) with a per-phase tooltip showing times and percentages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a Download CSV action to the frame results toolbar for query frames with response data. Query responses are flattened into rows (dot-notation for nested objects, '; '-joined scalar arrays, JSON-stringified object arrays, __block column for multi-block responses) and serialized as RFC-4180 CSV, downloaded as ratel-results-<YYYY-MM-DD-HHMM>.csv. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a -url-prefix flag (with RATEL_URL_PREFIX env fallback) to the Go server so Ratel can be hosted under a subpath behind reverse proxies, e.g. https://example.com/ratel/ (fixes dgraph-io#390). When a prefix is set: - all routes are served under the prefix via http.StripPrefix - the bare prefix redirects (301) to the prefix with a trailing slash - root-relative href/src URLs and the inline loader.js reference in the served index.html are rewritten to include the prefix - paths outside the prefix return 404 with a hint at the prefix With no prefix (the default) behavior is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three graph-view features on top of the sigma renderer: - Style rules (Neo4j Bloom-style): a Graph styles panel in the toolbar lists every group in the current result with a color picker and node size slider; overrides apply live via sigma reducers and persist in localStorage. lib/graphStyles.js (sanitize/persist/merge) has 11 unit tests. - Layout switcher: Force (ForceAtlas2 worker), Circular, and Packed (circlepack clustered by group) via a toolbar select; static layouts auto-fit the camera. - Legend filtering: clicking a predicate chip in the entity selector hides/shows that predicate's nodes and edges without re-querying; hidden chips render dimmed with strikethrough. Verified in a real browser against Dgraph v25: layouts switch, style panel renders per-group rows and applies color changes, legend chips toggle - zero page errors. Unit suite green, production build passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds an 'AI' button to the editor toolbar that opens a modal:
describe the query in plain language, get DQL back, review/edit it,
insert into the editor.
- Bring-your-own-key: Anthropic API key entered by the user, stored
in localStorage only; requests go directly browser -> model API
(anthropic-dangerous-direct-browser-access) so nothing routes
through the Ratel server. Model selectable (Haiku default, Sonnet).
- The current schema (schema {}) is summarized compactly (predicates
with types/indexes, type definitions, dgraph.* filtered out) and
sent as context, so generated queries use real predicates.
- The system prompt constrains output to a single fenced DQL block;
extraction falls back to raw text. Generated DQL is editable before
inserting.
- lib/nl2dql.js is pure and fully unit-tested (12 tests: settings
persistence/sanitization, schema summarization, prompt content,
extraction, API call shape, key/error handling).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The form's onSubmit handler called onLogin(userid, password) without the namespace argument, so keyboard-submitted logins on multi-tenancy clusters dispatched loginIntoNamespace with Number(undefined) = NaN instead of the namespace typed into the form. The Login button already passed it correctly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turns Ratel from a read-only viewer into an editor for scalar values (the headline feature of tools like G.V()): - Each attribute row in the node properties panel gets edit and delete actions: edit opens an inline input (number input for numbers, true/false select for booleans), Enter/check saves, Escape cancels; delete is two-step (trash -> 'sure?'). - '+ Add value' row sets a new predicate on the node. - Saves run single-triple N-Quad mutations (commitNow) through the existing dgraph client; values are written with the original value's type (xs:int / xs:float / xs:boolean, escaped strings) so edits never silently retype a predicate. Errors from the server surface in the panel; successful edits update the in-memory node. - lib/mutations.js validates uids (hex) and predicate names (no angle brackets/whitespace - no N-Quad injection), escapes string literals, and is fully unit-tested (11 tests). Mutation output verified against a live Dgraph v25: set string with escaped quotes, set typed int, delete specific/all values - all accepted and the final state matches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalizes the NL->DQL feature behind a provider registry: - Anthropic (Claude Haiku 4.5 / Sonnet 4.6), OpenAI (GPT-5 mini / GPT-5.1, chat completions with bearer auth) and Google Gemini (2.5 Flash / Pro, generateContent with the key in the x-goog-api-key header so it never appears in URLs). - Provider selector in the modal; API keys and model choice are stored per provider in localStorage. Legacy single-provider settings migrate into the anthropic slot automatically. - Each provider defines buildRequest/parseResponse/parseError; the generation flow, prompt and DQL extraction are shared. - 22 unit tests: request shapes for all three providers, response parsing, settings migration/sanitization, end-to-end mocks, error surfacing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a tab strip above the query editor so multiple queries can be drafted side by side. Tab state lives in the query reducer: the existing top-level fields remain the live (active-tab) fields, and switching tabs saves/restores them, so all existing consumers are untouched. Legacy persisted state without tabs hydrates into a single tab containing the current query. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Editor.scss paints .cm-invalidchar black on purpose: the graphql-ish editor mode flags valid DQL (dotted predicates like dgraph.type, numeric arguments) as invalid, and black-on-white makes those tokens read as normal text. On the dark background they were invisible. Mirror the intent under [data-theme='dark']: normal text color. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # client/src/components/EditorPanel.js
# Conflicts: # client/package.json
# Conflicts: # client/src/components/GraphContainer.js
Two UI fixes surfaced when the History button crowds the editor toolbar: - The toolbar used floats (.actions float left/right). When the right action group no longer fit beside Query/Mutate it dropped below, collapsing the float container into an empty band + a half-row of buttons. Switch .header/.actions to flexbox (flex-wrap, margin-left auto) so a crowded toolbar wraps into clean rows. - The history dropdown was position:absolute right:0 width:420px, anchored to the mid-toolbar History button — on a narrow editor pane it extended left off-screen and got clipped. Make it position:fixed with top/left/width computed from the button rect and clamped to the viewport, recomputed on resize, so it always stays visible. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add Louvain community detection and betweenness centrality (graphology ecosystem) as optional node rendering modes alongside the existing predicate-color / degree-size defaults: - Color by: Predicate (default) or Community - Size by: Degree (default), Centrality (betweenness), or Uniform Metrics are annotated onto the graphology graph in buildGraph and applied in the SigmaGraph node reducer, so default rendering is byte-for-byte unchanged. Betweenness is skipped above 1500 nodes to keep expansion responsive. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a path-finding mode to the graph toolbar. Toggle it on, click a source node then a target, and the shortest route between them (computed breadth-first over the currently rendered subgraph, edges treated as undirected) is highlighted while the rest of the graph dims back. A banner reports the hop count or that no path exists, with a Clear button. Path-finding runs client-side on the loaded graph, so it needs no extra server round-trip and works on exactly what the user can see. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a filter panel to the graph toolbar that hides nodes failing a connectivity (degree) range or an attribute predicate (contains / = / ≠ / > / < / exists). Edges drop out with either endpoint. The panel previews how many nodes are hidden and clears in one click. Filtering is applied in the SigmaGraph reducers via a precomputed hidden-node set, recomputed only when the filter spec or dataset changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Detect ISO datetime attributes on nodes and, when the dataset spans a range, offer a timeline control: a clock toggle reveals a scrubber with play/pause that reveals nodes as their earliest timestamp passes. Nodes without a time stay as structural context; edges drop with either hidden endpoint. The cutoff is applied in the SigmaGraph reducers against a node _time attribute annotated in buildGraph, so scrubbing/playback is a cheap refresh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # client/src/components/GraphContainer.js
Address review feedback on the latency bar: - The collapsed headline now shows the grand total (server phases + network), not parsing+processing+encoding, matching what users expect from "total". - Clicking the latency bar opens a 'Query latency breakdown' modal (the hover tooltip is kept) with one labelled, colored bar per phase plus a total row. - A 'Num UIDs' section counts the values each predicate contributed to the response (scalars once, child lists by length), a proxy for how much data the query returned and thus its processing/encoding/network cost. countPredicates/numUidSegments are pure and unit-tested. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The EditorTabs strip used hardcoded light colors, leaving a bright bar above the editor in dark mode. Add [data-theme=dark] overrides so the strip, inactive tabs, hover and rename input use the dark palette, and the active tab blends into the editor header (--bg-raised) below it. (EditorTabs ships on the multi-tab branch; these rules are dormant until both features are present, e.g. on sp/all-features.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per review, the Latency section now reads as a sequence over the total query duration: each phase bar starts where the previous phase ended (offset by cumulative elapsed time) instead of every bar starting at zero. Num UIDs bars stay left-aligned as plain counts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The modal chrome is themed centrally, but the latency/num-uids bar tracks rendered bright white and labels were dim in dark mode. Add [data-theme=dark] overrides (co-located with the component) so the empty tracks use the dark elevated surface, the total rule/border use the dark border, and labels/values stay legible. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 120px label column truncated 'Assign timestamp' to 'Assign timesta...'. Widen to 150px so the longest phase label fits without ellipsis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
variant='default' buttons (the 'Connected' status and 'Return to Ratel') rendered near-black on the dark modal surface. Give .btn-default the dark theme text color so they're readable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- GraphContainer: add type="button", aria-label, and title-bearing SVG icons to every toolbar button so the controls don't default to submit, expose themselves to screen readers, and avoid the noSvgWithoutTitle lint - EditorTabs: replace autoFocus on the rename input with a ref-based focus effect (also selects the existing name so renaming is faster) - SigmaGraph: drop the ambiguous `(this.draggedNode = null)` arrow body that read as a `return null`; use a block body so the assignment is the obvious side effect
…egree histogram Adds a read-only panel summarising the currently visible graph: node and edge counts, directed density, undirected connected components, reciprocity (fraction of edges with a reverse), degree statistics, a degree-distribution histogram, and the top hubs by degree and betweenness. The panel reuses the existing Sigma `buildGraph` so it sees the same graph topology the renderer draws. Stats are computed lazily — the graph is only built when the panel is open — and the betweenness ranking is skipped on graphs above the 1500-node threshold that `annotateMetrics` already enforces, so opening the panel never freezes the UI. Toggle sits in the graph toolbar next to the existing filter and style panels. The summary, the two top-N rankings, and the histogram all share the live dataset, so adding/removing nodes (e.g. via "Show more nodes") updates the panel automatically through the same `graphUpdateHack` signal the rest of the graph container already watches. `client/src/lib/graphStats.js` ships the stat helpers as a small, standalone module with 25 unit tests covering empty graphs, isolated components, partial reciprocity, and an even-degree median case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a read-only panel summarising the currently visible graph: node and edge counts, directed density, undirected connected components, reciprocity (fraction of edges with a reverse), degree statistics (avg/median/min/max), a degree-distribution histogram, and the top hubs by degree and betweenness.
The panel reuses the existing Sigma
buildGraphso it sees the same topology the renderer draws. Stats are computed lazily — the graph is only built when the panel is open — and the betweenness ranking is skipped on graphs above the 1500-node thresholdannotateMetricsalready enforces, so opening the panel never freezes the UI.Toggle sits in the graph toolbar next to the existing filter and style panels. Adding/removing nodes (e.g. via "Show more nodes") updates the panel automatically through the same
graphUpdateHacksignal the rest of the graph container already watches.What it shows
Files
client/src/lib/graphStats.js— pure-function stat helpers (summarizeGraph,topByAttribute, plus the per-stat primitives)client/src/lib/graphStats.test.js— 25 unit tests covering empty graphs, isolated components, partial reciprocity, and the even-degree median caseclient/src/components/GraphStatsPanel.js+.scss— slide-out panel (positioned likeGraphFilterPanel/GraphStylePanel), dark-mode aware via the existing[data-theme="dark"]hookclient/src/components/GraphContainer.js— toggle button + stats graph memoized on(nodesDataset, edgesDataset, graphUpdateHack, statsPanelOpen)Inspiration
Universally expected by every mature graph viz (Neo4j Browser, Memgraph Lab, Cytoscape, Gephi, yEd). Complements the existing community coloring, betweenness sizing, and shortest-path features without overlapping them — none of those surface a global overview.
Checklist
graphStats.test.js— 25 cases)241/241)trunk fmtclean (pre-commit hook verified)