feat(sdk): typed table sort keys in the dialog protocol (0.18.0)

CHANGELOG.md

@@ -5,6 +5,43 @@ All notable changes to `plotjuggler_sdk` are recorded here. Versioning policy is
 
 ## [0.18.0]
 
+### Feature: typed table sort keys — numeric columns sort numerically (MINOR)
+
+A table cell crossed the dialog protocol as text only, so the host could compare
+nothing but the rendered string and every numeric column sorted
+lexicographically (`720` before `7` before `65`). Sort keys now travel beside the
+display text. Backward-compatible JSON additions — no C ABI change,
+`PJ_DIALOG_PROTOCOL_VERSION` unchanged, `abi/baseline.abi` unchanged:
+
+- `PJ::TableItem { std::string text; std::optional<NumericValue> value; }` — the
+  display string and the ordering truth. Constructors cover a text cell, a
+  numeric cell (`std::to_string` rendering), and a numeric cell with plugin-owned
+  rendering (`TableItem(v, "5.60536e+08")`), which also expresses a *hidden* key:
+  display a date, sort on `int64` nanoseconds. `NumericValue` keeps the native
+  width, so `int64`/`uint64` values past 2⁵³ round-trip exactly rather than
+  degrading through a `double`.
+- `WidgetData::setTableRows(name, vector<vector<TableItem>>)` — an overload beside
+  the string one. Emits display text as the usual `rows` plus a **sparse**
+  `column_values` map (`{"<col>": [v0, v1, …]}`): only columns where some cell has
+  a value, with valueless cells as `null`, and the key omitted entirely when no
+  cell has one. Old hosts never look for it and read `rows` exactly as before.
+  Deriving both keys from one `TableItem` matrix means display and value cannot
+  desync.
+- `WidgetData::setTableSortIndicator(name, column, ascending)` → `sort_indicator`
+  — draws the header arrow for a table that sorts itself via `onHeaderClicked`,
+  which Qt otherwise leaves unpainted because its own sorting is off. Cosmetic
+  only.
+- `WidgetDataView::tableColumnValues(name)` / `tableSortIndicator(name)` for host
+  implementations. `tableColumnValues` drops a column whose value count disagrees
+  with `rows` or whose key is not a non-negative integer, rather than sorting some
+  rows by number and the rest by text.
+- `WidgetDataView::tableRows()` hardening: a non-string cell now yields an empty
+  string instead of being skipped. Skipping shifted every later cell one column
+  left and mis-aligned the row against its headers.
+- The plain-string `setTableRows` overload now erases any `column_values` a
+  previous typed delivery left on the same table, so alternating overloads can
+  never pair fresh rows with stale sort keys.
+
 ### Feature: batch table deltas for large QTableWidgets (MINOR)
 
 Mutate a table without resending the whole `rows` array (backward-compatible
@@ -24,7 +61,7 @@ JSON addition; no C ABI change, `PJ_DIALOG_PROTOCOL_VERSION` unchanged):
   omit-unchanged-fields pattern (with measured costs) and the delta ops.
 - SDK-side only: hosts apply `table_delta` from the companion PlotJuggler
   change onward; older hosts ignore the key (harmless no-op).
-  
+
 ### Feature: QDateTimeEdit event surface (MINOR)
 
 The dialog protocol's QDateTimeEdit setters (`setDateTime` / `setDateTimeRange`,

conanfile.py

@@ -6,7 +6,7 @@
   plugin_sdk   — umbrella for plugin authors (base + dialog SDK + parser SDK)
   plugin_host  — umbrella for host loaders (data_source/parser/toolbox/dialog)
 
-A consuming Conan recipe declares e.g. `plotjuggler_sdk/0.16.2` and then:
+A consuming Conan recipe declares e.g. `plotjuggler_sdk/0.18.0` and then:
 
     find_package(plotjuggler_sdk REQUIRED COMPONENTS plugin_sdk)
     target_link_libraries(my_plugin PRIVATE plotjuggler_sdk::plugin_sdk)
@@ -30,29 +30,6 @@
 
 class PlotjugglerSdkConan(ConanFile):
     name = "plotjuggler_sdk"
-    # 0.14.0 unified markers + transforms into the single host service
-    # `pj.data_processors.v1` via a `kind` discriminator (removed the old
-    # `pj.markers.v1` and the interim `pj.generators.v1`; generalized
-    # `create_data_processor`/`validate_data_processor_script` with
-    # kind/language/flags) and added the dialog-protocol additions (radio
-    # column + interactive sub-panel). Shipped as a MINOR bump rather than
-    # 1.0.0 because no public tag had ever carried `pj.data_processors.v1`,
-    # so no released plugin broke.
-    # 0.15.0 adds DataSource per-topic pause (advertise + demand-driven
-    # subscription) — strictly additive. See CHANGELOG.md.
-    # 0.16.0 removes the host-side PluginRuntimeCatalog (duplicate-resolution
-    # policy moved to the app, pj_runtime). No plugin links it and
-    # abi/baseline.abi is unchanged, so per the plugin-impact rule this is a
-    # MINOR, not a MAJOR. See CHANGELOG.md.
-    # 0.16.1 fixes plugin_data_api.hpp double formatting on Apple deployment
-    # targets older than macOS 13.3 (FP std::to_chars unavailable there) —
-    # PATCH, installed-header bug fix. See CHANGELOG.md.
-    # 0.17.0 extends the dialog protocol with backward-compatible additions
-    # (list_deletable / list_placeholder / chart_placeholder keys and the
-    # item_delete_index event) — MINOR. See CHANGELOG.md.
-    # 0.18.0 adds the QDateTimeEdit event surface (datetime_iso event,
-    # WidgetEvent::dateTimeChanged, DialogPluginTyped::onDateTimeChanged) —
-    # MINOR, header-only additions. See CHANGELOG.md.
     version = "0.18.0"
     # Apache-2.0 covers the whole SDK (pj_base + pj_plugins). See LICENSE.
     license = "Apache-2.0"

docs/dialog-sdk-reference.md

@@ -74,14 +74,21 @@ For the full tutorial, see [dialog-plugin-guide.md](../pj_plugins/docs/dialog-pl
 | Method | Description |
 |--------|-------------|
 | `setTableHeaders(name, vector<string>)` | Set column headers |
-| `setTableRows(name, vector<vector<string>>)` | Set row data |
+| `setTableRows(name, vector<vector<string>>)` | Set row data as plain text. Every column sorts **lexicographically** — `"9"` after `"10"`. |
+| `setTableRows(name, vector<vector<TableItem>>)` | Set row data with a per-cell sort key, so numeric columns sort numerically. `TableItem{text, optional<NumericValue>}`: `TableItem("x")` sorts by text, `TableItem(v)` renders and sorts on `v`, `TableItem(v, "display")` sorts on `v` but shows `display` (lossy or decorated rendering, or a hidden key such as a date over `int64` ns). Pass the native type — `NumericValue` keeps `int64`/`uint64` exact past 2⁵³. |
+| `setTableSortIndicator(name, column, ascending)` | Draw the header arrow **without** enabling Qt's sorting — for tables that sort themselves via `onHeaderClicked` (Qt paints an arrow only for its own sorting). Cosmetic; never reorders rows. |
 | `setSelectedRows(name, vector<int>)` | Set selected row indices |
 | `setDisabledRows(name, vector<int>)` | Grey out rows (non-selectable) |
 | `setTableRadioColumn(name, column, checked_row)` | Render `column` as an exclusive radio group; `checked_row` is selected (-1 = none). Fires `onTableRadioSelected`. |
 | `appendTableRows(name, seq, rows)` | Delta: append rows without resending `rows` (see guide → "Table deltas") |
 | `updateTableCells(name, seq, vector<TableCellUpdate>)` | Delta: rewrite individual cells (`{row, col, text}`, plugin row space) |
 | `removeTableRows(name, seq, vector<int>)` | Delta: remove plugin-space row indexes |
 
+> A table must not combine `sortingEnabled=true` in its `.ui` with
+> `onHeaderClicked` — Qt would sort the view while the plugin reorders the model,
+> and the plugin's order loses. See `pj_plugins/docs/dialog-plugin-guide.md` →
+> "Sortable Tables".
+
 ### QFrame Chart Container
 
 | Method | Description |

pj_plugins/dialog_protocol/CLAUDE.md

@@ -11,6 +11,13 @@ Local traps not visible from the headers:
   the XML, or the dialog renders with no OK/Cancel and no compile error.
 - `QTextEdit` / model-based `QTableView` are unsupported by the widget binding —
   use `QPlainTextEdit` / `QTableWidget`. See `../docs/dialog-plugin-guide.md`.
+- A table must not combine `sortingEnabled=true` in its `.ui` XML with
+  `onHeaderClicked`: Qt sorts the view while the plugin reorders the model and the
+  plugin's order loses. Sort keys (`setTableRows` with `TableItem`) or
+  `onHeaderClicked` — one per table, never both.
+- Sortable tables must also leave item drag/drop (`dragEnabled`, `InternalMove`)
+  OFF: Qt reconstructs dropped cells from serialized display roles, which strips
+  the typed sort key and leaves a column mixing keyed and keyless cells.
 - Headers here install into the SAME `pj_plugins/` include tree as the parent
   module (merged at install); keep names distinct.
 - `DialogHandle::borrowed()` / `fromBorrowed()` wrap a source/toolbox-owned

pj_plugins/dialog_protocol/include/pj_plugins/host/widget_data_view.hpp

@@ -113,13 +113,79 @@ class WidgetDataView {
       for (const auto& cell : row) {
         if (cell.is_string()) {
           cells.push_back(cell.get<std::string>());
+        } else {
+          // Keep the row's shape. Dropping an unrecognized cell would shift every
+          // column after it and silently mis-align the whole row.
+          cells.emplace_back();
         }
       }
       result.push_back(std::move(cells));
     }
     return result;
   }
 
+  /// Sparse per-column sort keys (see WidgetData::setTableRows for TableItem rows).
+  /// Maps column index -> one entry per row, in the same order as tableRows(). A
+  /// nullopt entry, or a column absent from the map, sorts by the cell's text.
+  /// Columns whose value count disagrees with `rows` are dropped: a partial key
+  /// column would sort some rows by number and others by text.
+  [[nodiscard]] std::map<int, std::vector<std::optional<NumericValue>>> tableColumnValues(std::string_view name) const {
+    std::map<int, std::vector<std::optional<NumericValue>>> result;
+    const nlohmann::json* w = widget(name);
+    if (!w) {
+      return result;
+    }
+    auto rows_it = w->find("rows");
+    const std::size_t row_count = (rows_it != w->end() && rows_it->is_array()) ? rows_it->size() : 0;
+    auto it = w->find("column_values");
+    if (it == w->end() || !it->is_object()) {
+      return result;
+    }
+    for (auto kv = it->begin(); kv != it->end(); ++kv) {
+      const nlohmann::json& values = kv.value();
+      const auto col = parseNumber<int>(kv.key());
+      if (!col.has_value() || *col < 0 || !values.is_array() || values.size() != row_count) {
+        continue;
+      }
+      std::vector<std::optional<NumericValue>> decoded;
+      decoded.reserve(values.size());
+      for (const auto& v : values) {
+        // The wire collapses every integer width to a JSON integer, so decode to the
+        // widest lossless alternative rather than guessing the plugin's original type.
+        if (v.is_number_unsigned()) {
+          decoded.emplace_back(NumericValue(v.get<uint64_t>()));
+        } else if (v.is_number_integer()) {
+          decoded.emplace_back(NumericValue(v.get<int64_t>()));
+        } else if (v.is_number_float()) {
+          decoded.emplace_back(NumericValue(v.get<double>()));
+        } else {
+          decoded.emplace_back(std::nullopt);
+        }
+      }
+      result.emplace(*col, std::move(decoded));
+    }
+    return result;
+  }
+
+  /// Sort arrow a plugin-sorted table asked the host to draw: {column, ascending}.
+  /// Purely cosmetic — it never changes row order (see setTableSortIndicator).
+  [[nodiscard]] std::optional<std::pair<int, bool>> tableSortIndicator(std::string_view name) const {
+    const nlohmann::json* w = widget(name);
+    if (!w) {
+      return std::nullopt;
+    }
+    auto it = w->find("sort_indicator");
+    if (it == w->end() || !it->is_object()) {
+      return std::nullopt;
+    }
+    auto col = it->find("col");
+    auto asc = it->find("asc");
+    if (col == it->end() || !col->is_number_integer() || asc == it->end() || !asc->is_boolean()) {
+      return std::nullopt;
+    }
+    return std::make_pair(col->get<int>(), asc->get<bool>());
+  }
+
   /// Column to render as an exclusive radio-button group (see setTableRadioColumn).
   [[nodiscard]] std::optional<int> tableRadioColumn(std::string_view name) const {
     return getInt(name, "radio_column");

pj_plugins/dialog_protocol/include/pj_plugins/sdk/widget_data.hpp

@@ -5,10 +5,16 @@
 #include <cstdint>
 #include <map>
 #include <nlohmann/json.hpp>
+#include <optional>
 #include <string>
 #include <string_view>
+#include <type_traits>
+#include <utility>
+#include <variant>
 #include <vector>
 
+#include "pj_base/types.hpp"
+
 namespace PJ {
 
 /// A single point in a chart series (used by setChartSeries).
@@ -17,6 +23,49 @@ struct ChartPoint {
   double y;
 };
 
+/// A table cell carrying both what it displays and what it sorts by.
+///
+/// `text` is rendered verbatim — the plugin owns formatting (`%g`, `%.2f`, units,
+/// "N/A"). `value` is the ordering truth. Without it the host can only compare the
+/// rendered string, so a numeric column sorts lexicographically ("9" lands after
+/// "10"); supplying it makes the host sort on the number instead.
+///
+/// `value` need not resemble `text`: `TableItem(ns_since_epoch, "2026-07-17 10:23")`
+/// displays a date and sorts on int64 nanoseconds. Leave it unset (the string
+/// constructors do) for text columns — those sort by `text`.
+///
+/// Pass the native type: `NumericValue` keeps each width exactly, so a uint64 count
+/// or an int64 nanosecond timestamp never round-trips through a double (which is
+/// only exact to 2^53).
+///
+/// Non-finite doubles (NaN, ±infinity) do not survive the JSON wire — dump()
+/// serializes them as null — so such a cell arrives at the host keyless and
+/// sorts in the text rank with the other keyless cells, not at the numeric
+/// extremes. Map infinities to a finite sentinel first if their ordering
+/// matters.
+struct TableItem {
+  std::string text;
+  std::optional<NumericValue> value;
+
+  TableItem() = default;
+  TableItem(std::string display) : text(std::move(display)) {}
+  TableItem(const char* display) : text(display) {}
+
+  /// Numeric cell rendered with the default representation of `v`. long double
+  /// is excluded up front — NumericValue has no alternative for it, and letting
+  /// it past the constraint would fail with an incomprehensible variant error.
+  template <
+      typename T, typename = std::enable_if_t<
+                      std::is_arithmetic_v<T> && !std::is_same_v<T, bool> && !std::is_same_v<T, long double>>>
+  TableItem(T v) : text(std::to_string(v)), value(NumericValue(v)) {}
+
+  /// Numeric cell with plugin-controlled rendering — `display` is shown, `v` is sorted on.
+  template <
+      typename T, typename = std::enable_if_t<
+                      std::is_arithmetic_v<T> && !std::is_same_v<T, bool> && !std::is_same_v<T, long double>>>
+  TableItem(T v, std::string display) : text(std::move(display)), value(NumericValue(v)) {}
+};
+
 /// A named series of XY points for chart display (used by setChartSeries).
 /// If `color` is non-empty (e.g. "#ff7f0e"), it overrides the built-in palette
 /// color for this series; otherwise the host's chart palette picks one.
@@ -243,7 +292,75 @@ class WidgetData {
   }
 
   WidgetData& setTableRows(std::string_view name, const std::vector<std::vector<std::string>>& rows) {
-    entry(name)["rows"] = rows;
+    auto& e = entry(name);
+    e["rows"] = rows;
+    // Plain rows carry no sort keys: drop any keys a previous TableItem delivery
+    // left behind, or — when the row count happens to match — the host would pair
+    // these rows with the STALE keys and silently sort them wrong.
+    e.erase("column_values");
+    return *this;
+  }
+
+  /// Rows that carry a sort key per cell (see TableItem). Emits the display text as
+  /// the usual `rows`, plus the keys as a sparse per-column map — so a host that
+  /// predates this overload still reads `rows` and behaves exactly as before.
+  /// Columns where no cell has a value are omitted entirely and sort by text.
+  WidgetData& setTableRows(std::string_view name, const std::vector<std::vector<TableItem>>& rows) {
+    auto& e = entry(name);
+
+    nlohmann::json text_rows = nlohmann::json::array();
+    std::size_t max_cols = 0;
+    for (const auto& row : rows) {
+      nlohmann::json cells = nlohmann::json::array();
+      for (const auto& item : row) {
+        cells.push_back(item.text);
+      }
+      text_rows.push_back(std::move(cells));
+      if (row.size() > max_cols) {
+        max_cols = row.size();
+      }
+    }
+    e["rows"] = std::move(text_rows);
+
+    nlohmann::json columns = nlohmann::json::object();
+    for (std::size_t c = 0; c < max_cols; ++c) {
+      bool any_value = false;
+      for (const auto& row : rows) {
+        if (c < row.size() && row[c].value.has_value()) {
+          any_value = true;
+          break;
+        }
+      }
+      if (!any_value) {
+        continue;
+      }
+      nlohmann::json values = nlohmann::json::array();
+      for (const auto& row : rows) {
+        if (c < row.size() && row[c].value.has_value()) {
+          // Push the native alternative: nlohmann stores int64/uint64 natively, so a
+          // large count or a nanosecond timestamp survives the round-trip exactly.
+          std::visit([&values](auto v) { values.push_back(v); }, *row[c].value);
+        } else {
+          values.push_back(nlohmann::json::value_t::null);
+        }
+      }
+      columns[std::to_string(c)] = std::move(values);
+    }
+    if (columns.empty()) {
+      e.erase("column_values");
+    } else {
+      e["column_values"] = std::move(columns);
+    }
+    return *this;
+  }
+
+  /// Draw the sort arrow on `column` WITHOUT enabling Qt's built-in sorting.
+  /// For tables that sort themselves via onHeaderClicked: Qt paints the indicator
+  /// only when its own sorting is on, so a plugin-sorted table shows no arrow
+  /// unless it declares one here. Re-send whenever the sort state changes.
+  WidgetData& setTableSortIndicator(std::string_view name, int column, bool ascending) {
+    auto& e = entry(name);
+    e["sort_indicator"] = {{"col", column}, {"asc", ascending}};
     return *this;
   }