diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d7287c..7373100 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,26 @@ All notable changes to `plotjuggler_sdk` are recorded here. Versioning policy is ## [0.18.0] +### Feature: batch table deltas for large QTableWidgets (MINOR) + +Mutate a table without resending the whole `rows` array (backward-compatible +JSON addition; no C ABI change, `PJ_DIALOG_PROTOCOL_VERSION` unchanged): + +- `WidgetData::appendTableRows` / `updateTableCells` / `removeTableRows` write + a per-widget `table_delta` object (`seq`, `append`, `update_cells`, + `remove_rows`). `seq` is plugin-owned; hosts apply a delta only when its seq + differs from the last one applied to that widget, in the order update_cells + → remove_rows → append, with all indexes addressing the pre-delta table. + New `TableCellUpdate` struct. +- `WidgetDataView::tableDelta()` returns the decoded `TableDeltaView` for host + implementations (strict: a malformed op rejects the whole delta; + `remove_rows` arrives descending and duplicate-free), with + `tableDeltaSeq()` as the cheap staleness pre-check. +- `dialog-plugin-guide.md` gains a "Large tables" section documenting the + omit-unchanged-fields pattern (with measured costs) and the delta ops. +- SDK-side only: hosts apply `table_delta` from the companion PlotJuggler + change onward; older hosts ignore the key (harmless no-op). + ### Feature: QDateTimeEdit event surface (MINOR) The dialog protocol's QDateTimeEdit setters (`setDateTime` / `setDateTimeRange`, diff --git a/docs/dialog-sdk-reference.md b/docs/dialog-sdk-reference.md index 82b9c86..bb91732 100644 --- a/docs/dialog-sdk-reference.md +++ b/docs/dialog-sdk-reference.md @@ -78,6 +78,9 @@ For the full tutorial, see [dialog-plugin-guide.md](../pj_plugins/docs/dialog-pl | `setSelectedRows(name, vector)` | Set selected row indices | | `setDisabledRows(name, vector)` | Grey out rows (non-selectable) | | `setTableRadioColumn(name, column, checked_row)` | Render `column` as an exclusive radio group; `checked_row` is selected (-1 = none). Fires `onTableRadioSelected`. | +| `appendTableRows(name, seq, rows)` | Delta: append rows without resending `rows` (see guide → "Table deltas") | +| `updateTableCells(name, seq, vector)` | Delta: rewrite individual cells (`{row, col, text}`, plugin row space) | +| `removeTableRows(name, seq, vector)` | Delta: remove plugin-space row indexes | ### QFrame Chart Container diff --git a/pj_plugins/dialog_protocol/include/pj_plugins/host/widget_data_view.hpp b/pj_plugins/dialog_protocol/include/pj_plugins/host/widget_data_view.hpp index 010d427..a979544 100644 --- a/pj_plugins/dialog_protocol/include/pj_plugins/host/widget_data_view.hpp +++ b/pj_plugins/dialog_protocol/include/pj_plugins/host/widget_data_view.hpp @@ -2,7 +2,9 @@ // Copyright 2026 Davide Faconti // SPDX-License-Identifier: Apache-2.0 +#include #include +#include #include #include #include @@ -127,6 +129,103 @@ class WidgetDataView { return getInt(name, "radio_checked_row"); } + /// Decoded batch table delta (see WidgetData::appendTableRows et al.). + /// Apply only when `seq` differs from the last seq applied to that widget, + /// in the order: update_cells, then remove_rows, then append. All indexes + /// address the table before the delta; `remove_rows` comes pre-normalized + /// (descending, duplicate-free) so it can be applied in vector order. + struct TableDeltaView { + std::uint64_t seq = 0; + std::vector> append; + struct CellUpdate { + int row = 0; + int col = 0; + std::string text; + }; + std::vector update_cells; + std::vector remove_rows; + }; + + /// The delta's seq alone, without decoding the (possibly large) ops — check + /// this against the last seq you applied before paying for tableDelta(). + [[nodiscard]] std::optional tableDeltaSeq(std::string_view name) const { + const nlohmann::json* w = widget(name); + if (!w) { + return std::nullopt; + } + auto it = w->find("table_delta"); + if (it == w->end() || !it->is_object()) { + return std::nullopt; + } + auto seq_it = it->find("seq"); + if (seq_it == it->end() || !seq_it->is_number_unsigned()) { + return std::nullopt; + } + return seq_it->get(); + } + + [[nodiscard]] std::optional tableDelta(std::string_view name) const { + auto seq = tableDeltaSeq(name); + if (!seq.has_value()) { + return std::nullopt; + } + const nlohmann::json* w = widget(name); + auto it = w->find("table_delta"); + // Strict decode: one malformed op rejects the whole delta. Partially + // applying and then treating the seq as consumed would leave the table + // diverged from the plugin's model with no way to repair it. + TableDeltaView delta; + delta.seq = *seq; + if (auto append_it = it->find("append"); append_it != it->end()) { + if (!append_it->is_array()) { + return std::nullopt; + } + for (const auto& row : *append_it) { + if (!row.is_array()) { + return std::nullopt; + } + std::vector cells; + cells.reserve(row.size()); + for (const auto& cell : row) { + if (!cell.is_string()) { + return std::nullopt; + } + cells.push_back(cell.get()); + } + delta.append.push_back(std::move(cells)); + } + } + if (auto cells_it = it->find("update_cells"); cells_it != it->end()) { + if (!cells_it->is_array()) { + return std::nullopt; + } + for (const auto& update : *cells_it) { + if (!update.is_array() || update.size() != 3 || !update[0].is_number_integer() || + !update[1].is_number_integer() || !update[2].is_string() || update[0].get() < 0 || + update[1].get() < 0) { + return std::nullopt; + } + delta.update_cells.push_back({update[0].get(), update[1].get(), update[2].get()}); + } + } + if (auto remove_it = it->find("remove_rows"); remove_it != it->end()) { + if (!remove_it->is_array()) { + return std::nullopt; + } + for (const auto& row : *remove_it) { + if (!row.is_number_integer() || row.get() < 0) { + return std::nullopt; + } + delta.remove_rows.push_back(row.get()); + } + // Normalized for direct application: descending and duplicate-free, so a + // host removing in vector order can never hit index shift. + std::sort(delta.remove_rows.begin(), delta.remove_rows.end(), std::greater<>()); + delta.remove_rows.erase(std::unique(delta.remove_rows.begin(), delta.remove_rows.end()), delta.remove_rows.end()); + } + return delta; + } + [[nodiscard]] std::optional> selectedRows(std::string_view name) const { const nlohmann::json* w = widget(name); if (!w) { diff --git a/pj_plugins/dialog_protocol/include/pj_plugins/sdk/widget_data.hpp b/pj_plugins/dialog_protocol/include/pj_plugins/sdk/widget_data.hpp index 6c206f1..fb74037 100644 --- a/pj_plugins/dialog_protocol/include/pj_plugins/sdk/widget_data.hpp +++ b/pj_plugins/dialog_protocol/include/pj_plugins/sdk/widget_data.hpp @@ -85,6 +85,15 @@ struct ChartMarker { return marks; } +/// One cell write of a table delta (see WidgetData::updateTableCells). +/// `row`/`col` are in the plugin's own row space — the same space +/// setTableRows / setSelectedRows use, regardless of any user sorting. +struct TableCellUpdate { + int row = 0; + int col = 0; + std::string text; +}; + /// One mark on a MarkerTimeline. `region` true → a resizable [start,end] span; /// false → a single-point event at `start` (end ignored). Positions are in the /// timeline's integer step domain (see setMarkerTimelineBounds). `id` is a stable @@ -291,6 +300,43 @@ class WidgetData { return *this; } + // --- Table deltas: mutate a large table without resending `rows` --- + // + // `seq` is plugin-owned; give every new delta a fresh value (a counter). + // The host applies a delta only when its seq DIFFERS from the last one it + // applied to that widget, so a full-state rebuild that still carries an + // already-applied delta is harmless (even with diffing disabled), and a + // restarted counter self-heals. All ops sent in one refresh must share one + // seq — a differing seq starts a fresh delta, discarding prior ops. The + // host applies: update_cells, then remove_rows, then append. All indexes + // address the table as it was BEFORE this delta (plugin row space, the same + // space setTableRows / setSelectedRows use) — ops cannot target rows this + // same delta appends. A `rows` field in the same refresh wins: the host + // applies the full replace and the delta counts as consumed. + + /// Append rows to the end of the table. + WidgetData& appendTableRows( + std::string_view name, std::uint64_t seq, const std::vector>& rows) { + tableDeltaEntry(name, seq)["append"] = rows; + return *this; + } + + /// Rewrite individual cells in place. + WidgetData& updateTableCells(std::string_view name, std::uint64_t seq, const std::vector& cells) { + auto arr = nlohmann::json::array(); + for (const auto& cell : cells) { + arr.push_back({cell.row, cell.col, cell.text}); + } + tableDeltaEntry(name, seq)["update_cells"] = std::move(arr); + return *this; + } + + /// Remove the given plugin-space row indexes. + WidgetData& removeTableRows(std::string_view name, std::uint64_t seq, const std::vector& rows) { + tableDeltaEntry(name, seq)["remove_rows"] = rows; + return *this; + } + /// Render column `column` of a QTableWidget as an exclusive radio-button group: /// one QRadioButton per row, only one on at a time. `checked_row` is the row /// whose radio is selected (-1 for none). The cells in `column` carry the radio @@ -705,6 +751,18 @@ class WidgetData { } return data_[key]; } + + /// The widget's table_delta object. All ops of one refresh must share one + /// seq: a differing seq starts a fresh delta, discarding prior ops, so + /// mixing seqs can never relabel an old op with a new sequence number. + nlohmann::json& tableDeltaEntry(std::string_view name, std::uint64_t seq) { + auto& delta = entry(name)["table_delta"]; + if (!delta.is_object() || !delta.contains("seq") || delta["seq"] != seq) { + delta = nlohmann::json::object(); + delta["seq"] = seq; + } + return delta; + } }; } // namespace PJ diff --git a/pj_plugins/dialog_protocol/tests/widget_data_test.cpp b/pj_plugins/dialog_protocol/tests/widget_data_test.cpp index d313abb..21d9c03 100644 --- a/pj_plugins/dialog_protocol/tests/widget_data_test.cpp +++ b/pj_plugins/dialog_protocol/tests/widget_data_test.cpp @@ -271,3 +271,32 @@ TEST(WidgetDataTest, SetCodeCaretTrackingExplicitFalse) { auto j = parse(wd); EXPECT_EQ(j["editor"]["code_caret_tracking"], false); } + +TEST(WidgetDataTest, TableDeltaOpsSerializeUnderOneKey) { + WidgetData wd; + wd.appendTableRows("tbl", 7, {{"a", "b"}, {"c", "d"}}); + wd.updateTableCells("tbl", 7, {{0, 1, "x"}}); + wd.removeTableRows("tbl", 7, {3, 5}); + auto j = parse(wd); + const auto& delta = j["tbl"]["table_delta"]; + EXPECT_EQ(delta["seq"], 7); + ASSERT_EQ(delta["append"].size(), 2U); + EXPECT_EQ(delta["append"][1][0], "c"); + ASSERT_EQ(delta["update_cells"].size(), 1U); + EXPECT_EQ(delta["update_cells"][0][0], 0); + EXPECT_EQ(delta["update_cells"][0][1], 1); + EXPECT_EQ(delta["update_cells"][0][2], "x"); + EXPECT_EQ(delta["remove_rows"], nlohmann::json({3, 5})); +} + +TEST(WidgetDataTest, TableDeltaDifferingSeqStartsFreshDelta) { + WidgetData wd; + wd.appendTableRows("tbl", 1, {{"old"}}); + wd.updateTableCells("tbl", 2, {{0, 0, "new"}}); + auto j = parse(wd); + const auto& delta = j["tbl"]["table_delta"]; + // The seq-1 append must not survive relabeled under seq 2. + EXPECT_EQ(delta["seq"], 2); + EXPECT_FALSE(delta.contains("append")); + ASSERT_EQ(delta["update_cells"].size(), 1U); +} diff --git a/pj_plugins/dialog_protocol/tests/widget_data_view_test.cpp b/pj_plugins/dialog_protocol/tests/widget_data_view_test.cpp index c2e45c6..a5453f8 100644 --- a/pj_plugins/dialog_protocol/tests/widget_data_view_test.cpp +++ b/pj_plugins/dialog_protocol/tests/widget_data_view_test.cpp @@ -304,3 +304,50 @@ TEST(WidgetDataViewTest, ChartPlaceholderRoundTrip) { ASSERT_TRUE(v.chartPlaceholder("chart").has_value()); EXPECT_EQ(*v.chartPlaceholder("chart"), "No data yet"); } + +TEST(WidgetDataViewTest, TableDeltaRoundTrip) { + PJ::WidgetData wd; + wd.appendTableRows("tbl", 9, {{"r1c1", "r1c2"}}); + wd.updateTableCells("tbl", 9, {{2, 0, "upd"}}); + wd.removeTableRows("tbl", 9, {1}); + PJ::WidgetDataView view(wd.toJson()); + auto delta = view.tableDelta("tbl"); + ASSERT_TRUE(delta.has_value()); + EXPECT_EQ(delta->seq, 9U); + ASSERT_EQ(delta->append.size(), 1U); + EXPECT_EQ(delta->append[0][1], "r1c2"); + ASSERT_EQ(delta->update_cells.size(), 1U); + EXPECT_EQ(delta->update_cells[0].row, 2); + EXPECT_EQ(delta->update_cells[0].col, 0); + EXPECT_EQ(delta->update_cells[0].text, "upd"); + EXPECT_EQ(delta->remove_rows, std::vector{1}); +} + +TEST(WidgetDataViewTest, TableDeltaAbsentYieldsNullopt) { + PJ::WidgetDataView view(R"({"tbl": {"rows": [["a"]]}})"); + EXPECT_FALSE(view.tableDelta("tbl").has_value()); +} + +TEST(WidgetDataViewTest, TableDeltaWithoutSeqYieldsNullopt) { + PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"append": [["a"]]}}})"); + EXPECT_FALSE(view.tableDelta("tbl").has_value()); +} + +TEST(WidgetDataViewTest, TableDeltaMalformedCellRejectsWholeDelta) { + // One malformed op poisons the delta: partial application would consume the + // seq while diverging from the plugin's model. + PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"seq": 3, "update_cells": [[0, "not-int", "x"]]}}})"); + EXPECT_FALSE(view.tableDelta("tbl").has_value()); +} + +TEST(WidgetDataViewTest, TableDeltaNegativeIndexRejectsWholeDelta) { + PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"seq": 3, "remove_rows": [1, -2]}}})"); + EXPECT_FALSE(view.tableDelta("tbl").has_value()); +} + +TEST(WidgetDataViewTest, TableDeltaRemoveRowsNormalizedDescendingUnique) { + PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"seq": 4, "remove_rows": [2, 7, 2, 5]}}})"); + auto delta = view.tableDelta("tbl"); + ASSERT_TRUE(delta.has_value()); + EXPECT_EQ(delta->remove_rows, (std::vector{7, 5, 2})); +} diff --git a/pj_plugins/docs/dialog-plugin-guide.md b/pj_plugins/docs/dialog-plugin-guide.md index c3e81ab..89478a2 100644 --- a/pj_plugins/docs/dialog-plugin-guide.md +++ b/pj_plugins/docs/dialog-plugin-guide.md @@ -626,6 +626,69 @@ and runs it modally. The sub-dialog supports standard `QDialogButtonBox` accept/reject wiring. This is a one-shot command — the request is consumed after opening the sub-dialog. +### Large tables — send only what changed + +Every field in the widget-data JSON is **optional**: a field the payload omits +applies no change. The naive pattern of rebuilding the full table in every +`widget_data()` call works, but it re-serializes and re-parses everything on +every refresh — order of magnitude, measured on a desktop Linux build at the +time of writing: a 10,000×6 table is ~1 MB of JSON and costs ~25 ms per full +refresh even when nothing changed, growing roughly linearly with row count +(~10× at 100k rows — visible jank). The host defends itself (per-key diffing +skips unchanged widgets, and byte-identical payloads are dropped after a cheap +string compare), but the plugin-side serialization can only be avoided by the +plugin. + +The efficient ladder, in order: + +1. **Send `rows` only when the data actually changed.** Track a dirty flag and + omit `setTableRows` otherwise — selection, colors, and visibility can be + sent alone. This alone removes the steady-state cost entirely. +2. **Filter with `setVisibleRows`** instead of resending a reduced `rows` + array — the host hides rows in place. +3. **Mutate with table deltas** for live feeds (see below) — append, update, + or remove rows without resending the array. +4. **Rethink beyond ~100k rows** — an item-based table with hundreds of + thousands of rows is the wrong UI; aggregate or page in the plugin. + +### Table deltas — `appendTableRows` / `updateTableCells` / `removeTableRows` + +Batch mutations for tables that change incrementally (streaming fault lists, +live status boards): + +```cpp +std::string widget_data() override { + PJ::WidgetData wd; + if (!pending_rows_.empty()) { + ++delta_seq_; + wd.appendTableRows("faultTable", delta_seq_, pending_rows_); + wd.updateTableCells("faultTable", delta_seq_, {{2, 1, "RESOLVED"}}); + pending_rows_.clear(); + } + return wd.toJson(); +} +``` + +`seq` is plugin-owned; give every new delta a fresh value (a simple counter). +The host applies a delta only when its seq **differs from the last one it +applied** to that widget, so a full-state rebuild that still carries an +already-applied delta is harmless, and a restarted counter self-heals. All ops +sharing one refresh share one seq (a differing seq starts a fresh delta, +discarding prior ops) and apply as: `update_cells`, then `remove_rows`, then +`append`. **All indexes address the table as it was before the delta** — the +plugin's own row space, the same space `setTableRows` and `setSelectedRows` +use, unaffected by user sorting; ops cannot target rows the same delta +appends. A delta with any malformed op is rejected whole (never partially +applied). Sending `rows` in the same refresh wins: the host applies the full +replace and the delta counts as consumed. + +> **Host support:** this SDK release ships the protocol (setters + the +> `WidgetDataView::tableDelta` decoder); the PlotJuggler host applies deltas +> from its companion release onward. Older hosts ignore the `table_delta` key +> entirely (unknown keys are skipped by design), so emitting deltas against +> one is a harmless no-op — keep a full `setTableRows` fallback if you must +> support them. + ## Choosing a Base Class | Class | Use when |