feat(dialog): batch table deltas — append/update/remove without resending rows

CHANGELOG.md

@@ -5,6 +5,26 @@ All notable changes to `plotjuggler_sdk` are recorded here. Versioning policy is
 
 ## [0.18.0]
 
+### Feature: batch table deltas for large QTableWidgets (MINOR)
+
+Mutate a table without resending the whole `rows` array (backward-compatible
+JSON addition; no C ABI change, `PJ_DIALOG_PROTOCOL_VERSION` unchanged):
+
+- `WidgetData::appendTableRows` / `updateTableCells` / `removeTableRows` write
+  a per-widget `table_delta` object (`seq`, `append`, `update_cells`,
+  `remove_rows`). `seq` is plugin-owned; hosts apply a delta only when its seq
+  differs from the last one applied to that widget, in the order update_cells
+  → remove_rows → append, with all indexes addressing the pre-delta table.
+  New `TableCellUpdate` struct.
+- `WidgetDataView::tableDelta()` returns the decoded `TableDeltaView` for host
+  implementations (strict: a malformed op rejects the whole delta;
+  `remove_rows` arrives descending and duplicate-free), with
+  `tableDeltaSeq()` as the cheap staleness pre-check.
+- `dialog-plugin-guide.md` gains a "Large tables" section documenting the
+  omit-unchanged-fields pattern (with measured costs) and the delta ops.
+- SDK-side only: hosts apply `table_delta` from the companion PlotJuggler
+  change onward; older hosts ignore the key (harmless no-op).
+
 ### Feature: QDateTimeEdit event surface (MINOR)
 
 The dialog protocol's QDateTimeEdit setters (`setDateTime` / `setDateTimeRange`,

docs/dialog-sdk-reference.md

@@ -78,6 +78,9 @@ For the full tutorial, see [dialog-plugin-guide.md](../pj_plugins/docs/dialog-pl
 | `setSelectedRows(name, vector<int>)` | Set selected row indices |
 | `setDisabledRows(name, vector<int>)` | Grey out rows (non-selectable) |
 | `setTableRadioColumn(name, column, checked_row)` | Render `column` as an exclusive radio group; `checked_row` is selected (-1 = none). Fires `onTableRadioSelected`. |
+| `appendTableRows(name, seq, rows)` | Delta: append rows without resending `rows` (see guide → "Table deltas") |
+| `updateTableCells(name, seq, vector<TableCellUpdate>)` | Delta: rewrite individual cells (`{row, col, text}`, plugin row space) |
+| `removeTableRows(name, seq, vector<int>)` | Delta: remove plugin-space row indexes |
 
 ### QFrame Chart Container
 

pj_plugins/dialog_protocol/include/pj_plugins/host/widget_data_view.hpp

@@ -2,7 +2,9 @@
 // Copyright 2026 Davide Faconti
 // SPDX-License-Identifier: Apache-2.0
 
+#include <algorithm>
 #include <cstdint>
+#include <functional>
 #include <map>
 #include <nlohmann/json.hpp>
 #include <optional>
@@ -127,6 +129,103 @@ class WidgetDataView {
     return getInt(name, "radio_checked_row");
   }
 
+  /// Decoded batch table delta (see WidgetData::appendTableRows et al.).
+  /// Apply only when `seq` differs from the last seq applied to that widget,
+  /// in the order: update_cells, then remove_rows, then append. All indexes
+  /// address the table before the delta; `remove_rows` comes pre-normalized
+  /// (descending, duplicate-free) so it can be applied in vector order.
+  struct TableDeltaView {
+    std::uint64_t seq = 0;
+    std::vector<std::vector<std::string>> append;
+    struct CellUpdate {
+      int row = 0;
+      int col = 0;
+      std::string text;
+    };
+    std::vector<CellUpdate> update_cells;
+    std::vector<int> remove_rows;
+  };
+
+  /// The delta's seq alone, without decoding the (possibly large) ops — check
+  /// this against the last seq you applied before paying for tableDelta().
+  [[nodiscard]] std::optional<std::uint64_t> tableDeltaSeq(std::string_view name) const {
+    const nlohmann::json* w = widget(name);
+    if (!w) {
+      return std::nullopt;
+    }
+    auto it = w->find("table_delta");
+    if (it == w->end() || !it->is_object()) {
+      return std::nullopt;
+    }
+    auto seq_it = it->find("seq");
+    if (seq_it == it->end() || !seq_it->is_number_unsigned()) {
+      return std::nullopt;
+    }
+    return seq_it->get<std::uint64_t>();
+  }
+
+  [[nodiscard]] std::optional<TableDeltaView> tableDelta(std::string_view name) const {
+    auto seq = tableDeltaSeq(name);
+    if (!seq.has_value()) {
+      return std::nullopt;
+    }
+    const nlohmann::json* w = widget(name);
+    auto it = w->find("table_delta");
+    // Strict decode: one malformed op rejects the whole delta. Partially
+    // applying and then treating the seq as consumed would leave the table
+    // diverged from the plugin's model with no way to repair it.
+    TableDeltaView delta;
+    delta.seq = *seq;
+    if (auto append_it = it->find("append"); append_it != it->end()) {
+      if (!append_it->is_array()) {
+        return std::nullopt;
+      }
+      for (const auto& row : *append_it) {
+        if (!row.is_array()) {
+          return std::nullopt;
+        }
+        std::vector<std::string> cells;
+        cells.reserve(row.size());
+        for (const auto& cell : row) {
+          if (!cell.is_string()) {
+            return std::nullopt;
+          }
+          cells.push_back(cell.get<std::string>());
+        }
+        delta.append.push_back(std::move(cells));
+      }
+    }
+    if (auto cells_it = it->find("update_cells"); cells_it != it->end()) {
+      if (!cells_it->is_array()) {
+        return std::nullopt;
+      }
+      for (const auto& update : *cells_it) {
+        if (!update.is_array() || update.size() != 3 || !update[0].is_number_integer() ||
+            !update[1].is_number_integer() || !update[2].is_string() || update[0].get<int>() < 0 ||
+            update[1].get<int>() < 0) {
+          return std::nullopt;
+        }
+        delta.update_cells.push_back({update[0].get<int>(), update[1].get<int>(), update[2].get<std::string>()});
+      }
+    }
+    if (auto remove_it = it->find("remove_rows"); remove_it != it->end()) {
+      if (!remove_it->is_array()) {
+        return std::nullopt;
+      }
+      for (const auto& row : *remove_it) {
+        if (!row.is_number_integer() || row.get<int>() < 0) {
+          return std::nullopt;
+        }
+        delta.remove_rows.push_back(row.get<int>());
+      }
+      // Normalized for direct application: descending and duplicate-free, so a
+      // host removing in vector order can never hit index shift.
+      std::sort(delta.remove_rows.begin(), delta.remove_rows.end(), std::greater<>());
+      delta.remove_rows.erase(std::unique(delta.remove_rows.begin(), delta.remove_rows.end()), delta.remove_rows.end());
+    }
+    return delta;
+  }
+
   [[nodiscard]] std::optional<std::vector<int>> selectedRows(std::string_view name) const {
     const nlohmann::json* w = widget(name);
     if (!w) {

pj_plugins/dialog_protocol/include/pj_plugins/sdk/widget_data.hpp

@@ -85,6 +85,15 @@ struct ChartMarker {
   return marks;
 }
 
+/// One cell write of a table delta (see WidgetData::updateTableCells).
+/// `row`/`col` are in the plugin's own row space — the same space
+/// setTableRows / setSelectedRows use, regardless of any user sorting.
+struct TableCellUpdate {
+  int row = 0;
+  int col = 0;
+  std::string text;
+};
+
 /// One mark on a MarkerTimeline. `region` true → a resizable [start,end] span;
 /// false → a single-point event at `start` (end ignored). Positions are in the
 /// timeline's integer step domain (see setMarkerTimelineBounds). `id` is a stable
@@ -291,6 +300,43 @@ class WidgetData {
     return *this;
   }
 
+  // --- Table deltas: mutate a large table without resending `rows` ---
+  //
+  // `seq` is plugin-owned; give every new delta a fresh value (a counter).
+  // The host applies a delta only when its seq DIFFERS from the last one it
+  // applied to that widget, so a full-state rebuild that still carries an
+  // already-applied delta is harmless (even with diffing disabled), and a
+  // restarted counter self-heals. All ops sent in one refresh must share one
+  // seq — a differing seq starts a fresh delta, discarding prior ops. The
+  // host applies: update_cells, then remove_rows, then append. All indexes
+  // address the table as it was BEFORE this delta (plugin row space, the same
+  // space setTableRows / setSelectedRows use) — ops cannot target rows this
+  // same delta appends. A `rows` field in the same refresh wins: the host
+  // applies the full replace and the delta counts as consumed.
+
+  /// Append rows to the end of the table.
+  WidgetData& appendTableRows(
+      std::string_view name, std::uint64_t seq, const std::vector<std::vector<std::string>>& rows) {
+    tableDeltaEntry(name, seq)["append"] = rows;
+    return *this;
+  }
+
+  /// Rewrite individual cells in place.
+  WidgetData& updateTableCells(std::string_view name, std::uint64_t seq, const std::vector<TableCellUpdate>& cells) {
+    auto arr = nlohmann::json::array();
+    for (const auto& cell : cells) {
+      arr.push_back({cell.row, cell.col, cell.text});
+    }
+    tableDeltaEntry(name, seq)["update_cells"] = std::move(arr);
+    return *this;
+  }
+
+  /// Remove the given plugin-space row indexes.
+  WidgetData& removeTableRows(std::string_view name, std::uint64_t seq, const std::vector<int>& rows) {
+    tableDeltaEntry(name, seq)["remove_rows"] = rows;
+    return *this;
+  }
+
   /// Render column `column` of a QTableWidget as an exclusive radio-button group:
   /// one QRadioButton per row, only one on at a time. `checked_row` is the row
   /// whose radio is selected (-1 for none). The cells in `column` carry the radio
@@ -705,6 +751,18 @@ class WidgetData {
     }
     return data_[key];
   }
+
+  /// The widget's table_delta object. All ops of one refresh must share one
+  /// seq: a differing seq starts a fresh delta, discarding prior ops, so
+  /// mixing seqs can never relabel an old op with a new sequence number.
+  nlohmann::json& tableDeltaEntry(std::string_view name, std::uint64_t seq) {
+    auto& delta = entry(name)["table_delta"];
+    if (!delta.is_object() || !delta.contains("seq") || delta["seq"] != seq) {
+      delta = nlohmann::json::object();
+      delta["seq"] = seq;
+    }
+    return delta;
+  }
 };
 
 }  // namespace PJ

pj_plugins/dialog_protocol/tests/widget_data_test.cpp

@@ -271,3 +271,32 @@ TEST(WidgetDataTest, SetCodeCaretTrackingExplicitFalse) {
   auto j = parse(wd);
   EXPECT_EQ(j["editor"]["code_caret_tracking"], false);
 }
+
+TEST(WidgetDataTest, TableDeltaOpsSerializeUnderOneKey) {
+  WidgetData wd;
+  wd.appendTableRows("tbl", 7, {{"a", "b"}, {"c", "d"}});
+  wd.updateTableCells("tbl", 7, {{0, 1, "x"}});
+  wd.removeTableRows("tbl", 7, {3, 5});
+  auto j = parse(wd);
+  const auto& delta = j["tbl"]["table_delta"];
+  EXPECT_EQ(delta["seq"], 7);
+  ASSERT_EQ(delta["append"].size(), 2U);
+  EXPECT_EQ(delta["append"][1][0], "c");
+  ASSERT_EQ(delta["update_cells"].size(), 1U);
+  EXPECT_EQ(delta["update_cells"][0][0], 0);
+  EXPECT_EQ(delta["update_cells"][0][1], 1);
+  EXPECT_EQ(delta["update_cells"][0][2], "x");
+  EXPECT_EQ(delta["remove_rows"], nlohmann::json({3, 5}));
+}
+
+TEST(WidgetDataTest, TableDeltaDifferingSeqStartsFreshDelta) {
+  WidgetData wd;
+  wd.appendTableRows("tbl", 1, {{"old"}});
+  wd.updateTableCells("tbl", 2, {{0, 0, "new"}});
+  auto j = parse(wd);
+  const auto& delta = j["tbl"]["table_delta"];
+  // The seq-1 append must not survive relabeled under seq 2.
+  EXPECT_EQ(delta["seq"], 2);
+  EXPECT_FALSE(delta.contains("append"));
+  ASSERT_EQ(delta["update_cells"].size(), 1U);
+}

pj_plugins/dialog_protocol/tests/widget_data_view_test.cpp

@@ -304,3 +304,50 @@ TEST(WidgetDataViewTest, ChartPlaceholderRoundTrip) {
   ASSERT_TRUE(v.chartPlaceholder("chart").has_value());
   EXPECT_EQ(*v.chartPlaceholder("chart"), "No data yet");
 }
+
+TEST(WidgetDataViewTest, TableDeltaRoundTrip) {
+  PJ::WidgetData wd;
+  wd.appendTableRows("tbl", 9, {{"r1c1", "r1c2"}});
+  wd.updateTableCells("tbl", 9, {{2, 0, "upd"}});
+  wd.removeTableRows("tbl", 9, {1});
+  PJ::WidgetDataView view(wd.toJson());
+  auto delta = view.tableDelta("tbl");
+  ASSERT_TRUE(delta.has_value());
+  EXPECT_EQ(delta->seq, 9U);
+  ASSERT_EQ(delta->append.size(), 1U);
+  EXPECT_EQ(delta->append[0][1], "r1c2");
+  ASSERT_EQ(delta->update_cells.size(), 1U);
+  EXPECT_EQ(delta->update_cells[0].row, 2);
+  EXPECT_EQ(delta->update_cells[0].col, 0);
+  EXPECT_EQ(delta->update_cells[0].text, "upd");
+  EXPECT_EQ(delta->remove_rows, std::vector<int>{1});
+}
+
+TEST(WidgetDataViewTest, TableDeltaAbsentYieldsNullopt) {
+  PJ::WidgetDataView view(R"({"tbl": {"rows": [["a"]]}})");
+  EXPECT_FALSE(view.tableDelta("tbl").has_value());
+}
+
+TEST(WidgetDataViewTest, TableDeltaWithoutSeqYieldsNullopt) {
+  PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"append": [["a"]]}}})");
+  EXPECT_FALSE(view.tableDelta("tbl").has_value());
+}
+
+TEST(WidgetDataViewTest, TableDeltaMalformedCellRejectsWholeDelta) {
+  // One malformed op poisons the delta: partial application would consume the
+  // seq while diverging from the plugin's model.
+  PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"seq": 3, "update_cells": [[0, "not-int", "x"]]}}})");
+  EXPECT_FALSE(view.tableDelta("tbl").has_value());
+}
+
+TEST(WidgetDataViewTest, TableDeltaNegativeIndexRejectsWholeDelta) {
+  PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"seq": 3, "remove_rows": [1, -2]}}})");
+  EXPECT_FALSE(view.tableDelta("tbl").has_value());
+}
+
+TEST(WidgetDataViewTest, TableDeltaRemoveRowsNormalizedDescendingUnique) {
+  PJ::WidgetDataView view(R"({"tbl": {"table_delta": {"seq": 4, "remove_rows": [2, 7, 2, 5]}}})");
+  auto delta = view.tableDelta("tbl");
+  ASSERT_TRUE(delta.has_value());
+  EXPECT_EQ(delta->remove_rows, (std::vector<int>{7, 5, 2}));
+}

pj_plugins/docs/dialog-plugin-guide.md

@@ -626,6 +626,69 @@ and runs it modally. The sub-dialog supports standard `QDialogButtonBox`
 accept/reject wiring. This is a one-shot command — the request is consumed
 after opening the sub-dialog.
 
+### Large tables — send only what changed
+
+Every field in the widget-data JSON is **optional**: a field the payload omits
+applies no change. The naive pattern of rebuilding the full table in every
+`widget_data()` call works, but it re-serializes and re-parses everything on
+every refresh — order of magnitude, measured on a desktop Linux build at the
+time of writing: a 10,000×6 table is ~1 MB of JSON and costs ~25 ms per full
+refresh even when nothing changed, growing roughly linearly with row count
+(~10× at 100k rows — visible jank). The host defends itself (per-key diffing
+skips unchanged widgets, and byte-identical payloads are dropped after a cheap
+string compare), but the plugin-side serialization can only be avoided by the
+plugin.
+
+The efficient ladder, in order:
+
+1. **Send `rows` only when the data actually changed.** Track a dirty flag and
+   omit `setTableRows` otherwise — selection, colors, and visibility can be
+   sent alone. This alone removes the steady-state cost entirely.
+2. **Filter with `setVisibleRows`** instead of resending a reduced `rows`
+   array — the host hides rows in place.
+3. **Mutate with table deltas** for live feeds (see below) — append, update,
+   or remove rows without resending the array.
+4. **Rethink beyond ~100k rows** — an item-based table with hundreds of
+   thousands of rows is the wrong UI; aggregate or page in the plugin.
+
+### Table deltas — `appendTableRows` / `updateTableCells` / `removeTableRows`
+
+Batch mutations for tables that change incrementally (streaming fault lists,
+live status boards):
+
+```cpp
+std::string widget_data() override {
+  PJ::WidgetData wd;
+  if (!pending_rows_.empty()) {
+    ++delta_seq_;
+    wd.appendTableRows("faultTable", delta_seq_, pending_rows_);
+    wd.updateTableCells("faultTable", delta_seq_, {{2, 1, "RESOLVED"}});
+    pending_rows_.clear();
+  }
+  return wd.toJson();
+}
+```
+
+`seq` is plugin-owned; give every new delta a fresh value (a simple counter).
+The host applies a delta only when its seq **differs from the last one it
+applied** to that widget, so a full-state rebuild that still carries an
+already-applied delta is harmless, and a restarted counter self-heals. All ops
+sharing one refresh share one seq (a differing seq starts a fresh delta,
+discarding prior ops) and apply as: `update_cells`, then `remove_rows`, then
+`append`. **All indexes address the table as it was before the delta** — the
+plugin's own row space, the same space `setTableRows` and `setSelectedRows`
+use, unaffected by user sorting; ops cannot target rows the same delta
+appends. A delta with any malformed op is rejected whole (never partially
+applied). Sending `rows` in the same refresh wins: the host applies the full
+replace and the delta counts as consumed.
+
+> **Host support:** this SDK release ships the protocol (setters + the
+> `WidgetDataView::tableDelta` decoder); the PlotJuggler host applies deltas
+> from its companion release onward. Older hosts ignore the `table_delta` key
+> entirely (unknown keys are skipped by design), so emitting deltas against
+> one is a harmless no-op — keep a full `setTableRows` fallback if you must
+> support them.
+
 ## Choosing a Base Class
 
 | Class | Use when |