Rewrite dissector: modular sources, dual-protocol, version-tested

.github/workflows/dist.yml

@@ -0,0 +1,50 @@
+name: dist
+
+# Verify the committed dist/ builds are exactly what amalgamate.sh produces from
+# the current src/. Without this, an edit to src/ could ship while dist/ (what
+# users actually load into Wireshark) silently lags behind.
+
+on:
+  push:
+    branches: [master]
+    paths:
+      - "src/**"
+      - "dist/**"
+      - "MessagePack.lua"
+      - "amalgamate.sh"
+      - ".github/workflows/dist.yml"
+  pull_request:
+    paths:
+      - "src/**"
+      - "dist/**"
+      - "MessagePack.lua"
+      - "amalgamate.sh"
+      - ".github/workflows/dist.yml"
+
+jobs:
+  check-dist:
+    name: dist/ is up to date
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Regenerate dist/ from src/
+        run: sh ./amalgamate.sh
+
+      - name: Fail if dist/ differs from the committed output
+        run: |
+          if [ -n "$(git status --porcelain -- dist)" ]; then
+            echo "::error::dist/ is stale — run ./amalgamate.sh and commit the result."
+            git --no-pager diff -- dist
+            git status --porcelain -- dist
+            exit 1
+          fi
+          echo "dist/ matches the current src/."
+
+      - name: Syntax-check the generated dissectors
+        run: |
+          sudo apt-get update -qq
+          sudo apt-get install -y -qq lua5.4
+          for f in dist/*.lua; do
+            luac5.4 -p "$f" && echo "ok: $f"
+          done

.github/workflows/tests.yml

@@ -0,0 +1,59 @@
+name: tests
+
+# Decode the captured fixtures (tests/pcap/) with the dissector and assert each
+# decodes cleanly and covers the request types expected for its Tarantool
+# version. Guards against regressions in either protocol path.
+
+on:
+  push:
+    branches: [master]
+    paths:
+      - "src/**"
+      - "dist/**"
+      - "tests/**"
+      - "MessagePack.lua"
+      - "amalgamate.sh"
+      - ".github/workflows/tests.yml"
+  pull_request:
+    paths:
+      - "src/**"
+      - "dist/**"
+      - "tests/**"
+      - "MessagePack.lua"
+      - "amalgamate.sh"
+      - ".github/workflows/tests.yml"
+
+jobs:
+  dissect:
+    name: ${{ matrix.name }}
+    runs-on: ubuntu-24.04
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - name: Wireshark 4.x (noble)
+            jammy: false
+          - name: Wireshark 3.x (jammy)
+            jammy: true
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install tshark
+        env:
+          DEBIAN_FRONTEND: noninteractive
+        run: |
+          if ${{ matrix.jammy }}; then
+            echo "deb http://archive.ubuntu.com/ubuntu jammy main universe" \
+              | sudo tee /etc/apt/sources.list.d/jammy.list
+            sudo apt-get update -qq
+            sudo apt-get install -y -qq -t jammy tshark
+          else
+            sudo apt-get update -qq
+            sudo apt-get install -y -qq tshark
+          fi
+
+      - name: Show tshark version
+        run: tshark --version | head -1
+
+      - name: Decode fixtures with the dissector
+        run: sh tests/run.sh

README.md

@@ -10,32 +10,130 @@ dissector implemented for Tarantool binary protocol.
 
 ![Wireshark][screenshot]
 
+The dissector auto-detects the wire format per PDU by inspecting the leading
+bytes: the modern MsgPack IProto (Tarantool 1.6–3.x, always framed with a 5-byte
+`0xce` length prefix) and the legacy ≤1.5 binary protocol (a fixed 12-byte
+little-endian `<type><len><req_id>` header). It loops over every PDU in a
+segment, so it also decodes a mixed capture where old and new clients share one
+port (e.g. legacy `box.dostring` calls alongside MsgPack `call_16`/`ping`).
+
+Three ready-to-use, self-contained single-file builds are generated under
+`dist/` (MsgPack is bundled in; nothing else to copy). Pick the one matching your
+traffic — load only one at a time, they share the protocol name:
+
+| File | Covers |
+| --- | --- |
+| [`dist/tarantool.dissector.lua`][dist-all] | **all versions** — modern + legacy, auto-detected (recommended) |
+| [`dist/tarantool-modern.dissector.lua`][dist-modern] | modern MsgPack IProto only (Tarantool 1.6–3.x) |
+| [`dist/tarantool-legacy.dissector.lua`][dist-legacy] | legacy pre-MsgPack only (Tarantool ≤1.5) |
+
+For the modern format it understands the current IProto request set, including
+SQL (`execute`, `prepare`), interactive transactions over streams (`begin`,
+`commit`, `rollback`), protocol negotiation (`id`), event watchers (`watch`,
+`unwatch`, `event`, `watch_once`) and the structured (`MP_ERROR`) error format.
+Tuple values encoded as MsgPack extensions are decoded too: `decimal`, `uuid`,
+`datetime` and `interval` render as real values, while opaque extensions
+(`error`, `compression`, `tuple`, `arrow`) show as a labelled blob. Unsigned
+64-bit values are rendered unsigned, and it handles TCP reassembly of large
+packets and several pipelined packets in a single segment.
+
+The following display filter fields are available:
+
+| Field | Description |
+| --- | --- |
+| `tnt.type` | request/response code, e.g. `tnt.type == 0x01` for selects |
+| `tnt.request` | request name, e.g. `tnt.request == "call"` |
+| `tnt.sync` | request id, handy to match a response to its request |
+| `tnt.schema_version` | schema version reported in responses |
+| `tnt.stream_id` | stream id of an interactive transaction |
+| `tnt.response` | `true` for responses, `false` for requests |
+
+### Installation
+
+The `dist/` builds are self-contained — one file, no dependencies to copy. Put
+it in Wireshark's **Personal Lua Plugins** folder.
+
+1. Find the folder. Run `tshark -G folders` (or *Help → About Wireshark →
+   Folders* in the GUI) and look for the `Personal Lua Plugins` line. Typical
+   locations:
+
+   | OS | Path |
+   | --- | --- |
+   | Linux / macOS | `~/.local/lib/wireshark/plugins` |
+   | Windows | `%APPDATA%\Wireshark\plugins` |
+
+2. Copy the build you want there. **Rename it so the filename has no extra dot
+   before `.lua`.** Wireshark treats a dot in a plugin filename as a module path,
+   so `tarantool.dissector.lua` would be looked up as the module
+   `tarantool/dissector.lua` and fail to auto-load. Install it as `tarantool.lua`:
+
+   ```sh
+   DEST=~/.local/lib/wireshark/plugins
+   mkdir -p "$DEST"
+   cp dist/tarantool.dissector.lua "$DEST/tarantool.lua"
+   ```
+
+   A second Tarantool dissector registering the same protocol name fails to load
+   with *"there cannot be two protocols with the same description"*, so keep only
+   one in that folder.
+
+3. Load it. Restart Wireshark, or in a running GUI reload plugins with
+   *Analyze → Reload Lua Plugins* (**Ctrl+Shift+L**, **⌘⇧L** on macOS).
+
+Alternatively, skip installation and pass the dissector ad-hoc on the command
+line:
+
+```sh
+wireshark -X lua_script:dist/tarantool.dissector.lua
+tshark    -X lua_script:dist/tarantool.dissector.lua -V -r capture.pcap
+```
+
+### Building from source
+
+The `dist/` files are generated; the real source lives under `src/` as small
+modules, combined by `amalgamate.sh` (POSIX shell, no toolchain needed):
+
+| Path | Role |
+| --- | --- |
+| `src/core.lua` | Proto, ProtoFields, port pref, greeting, main loop, registration |
+| `src/msgpack_ext.lua` | MsgPack ext decoding (decimal/uuid/datetime/interval) + unsigned-64 fix |
+| `src/modern.lua` | modern MsgPack IProto constants + decoders |
+| `src/legacy.lua` | legacy ≤1.5 framing + decoders |
+| `src/entry_*.lua` | per-build entry points wiring the dispatch |
+| `MessagePack.lua` | vendored pure-Lua MsgPack (inlined into the modern/all builds) |
+
+Each build inlines the modules it needs via `package.preload`, so the modules
+stay independent (and `require`-able) while the output is one self-contained
+file. Edit `src/` and regenerate all three builds with:
+
+```sh
+./amalgamate.sh
+```
+
 ### How to use
 
-- Setup Wireshark. See chapter [Building and Installing
-  Wireshark][building-and-installing-wireshark] in documentation.
-- Put a Lua file with dissector and `MessagePack.lua` to a directory with
-  plugins for Wireshark, directory depends on operating system, please refer to
-  chapter [Plugin folders][plugin-folders].
-  Note that Wireshark requires root privileges, make sure you are using plugin
-  directory for a user that is used for running Wireshark. It possible to run
-  Wireshark in terminal and pass Lua extension explicitly: `wireshark -X
-  lua_script:tarantool.dissector.lua` or `tshark -X
-  lua_script:tarantool.dissector.lua -V`.
-- If for some reason you still use Tarantool <= 1.5, use `tarantool15.dissector.lua`
-- Run Wireshark. By default Tarantool protocol dissector decodes TCP packets
-  with port 3301. However one can change a port for dissector in Wireshark
-  settings, see chapter [Control Protocol dissection][control-protocol-dissection].
-
-### How to test
-
-There is a script `test.lua` that uses Tarantool instance remotely via network
-and covers most part of IProto commands. For testing one can run Wireshark on
-local interface `lo0` with filtering by port 3301 and run script with command
-`tarantool test.lua`.
-
-[box-protocol]: https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/
+By default the dissector decodes TCP packets on port 3301. The port is
+configurable in *Edit → Preferences → Protocols → Tarantool*, or apply it to a
+single conversation via *Decode As…*, see chapter [Control Protocol
+dissection][control-protocol-dissection]. Capturing requires permission to read
+from the network interface (root, or membership in a capture group such as
+`access_bpf` on macOS / `wireshark` on Linux).
+
+Legacy ≤1.5 servers default to a different binary port (33013). Since the
+dissector only auto-attaches to the configured port (3301), point it at the
+legacy port too — set the Tarantool port preference to 33013, or use *Decode
+As… → tcp.port 33013 → Tarantool*. (This is why the test suite passes
+`-d tcp.port==33013,tarantool` for the 1.5 capture.)
+
+### Tests
+
+The dissector is verified against captures from real servers of several
+Tarantool versions, and traffic can be regenerated by hand. See
+[TESTING.md](TESTING.md).
+
+[box-protocol]: https://www.tarantool.io/en/doc/latest/reference/internals/box_protocol/
 [screenshot]: screenshot.png
-[building-and-installing-wireshark]: https://www.wireshark.org/docs/wsug_html_chunked/ChapterBuildInstall.html
-[plugin-folders]: https://www.wireshark.org/docs/wsug_html_chunked/ChPluginFolders.html
 [control-protocol-dissection]: https://www.wireshark.org/docs/wsug_html_chunked/ChCustProtocolDissectionSection.html
+[dist-all]: https://raw.githubusercontent.com/tarantool/tarantool-dissector/master/dist/tarantool.dissector.lua
+[dist-modern]: https://raw.githubusercontent.com/tarantool/tarantool-dissector/master/dist/tarantool-modern.dissector.lua
+[dist-legacy]: https://raw.githubusercontent.com/tarantool/tarantool-dissector/master/dist/tarantool-legacy.dissector.lua

TESTING.md

@@ -0,0 +1,40 @@
+## Testing
+
+`tests/pcap/` holds captures from real servers of several Tarantool versions,
+each exercising the functionality that version supports:
+
+| Fixture | Captured from | Covers |
+| --- | --- | --- |
+| `tarantool-1.5.pcap` | Tarantool 1.5 (legacy binary, port 33013) | insert, select, update, delete, call |
+| `tarantool-1.10.pcap` | Tarantool 1.10 (MsgPack) | auth, ping, CRUD, call, eval |
+| `tarantool-2.11.pcap` | Tarantool 2.11 (MsgPack) | the above + SQL, streams (begin/commit/rollback), `id`, watchers, error stack |
+| `tarantool-3.x.pcap` | Tarantool 3.x (MsgPack) | the above + `watch_once` and MsgPack ext types (decimal, uuid, datetime, interval) |
+| `tarantool-combined.pcap` | 1.5 + 3.x merged | both framings in one capture — exercises the per-PDU legacy/modern dispatch |
+| `tarantool-replication-async.pcap` | 3-node master-master, ports 3311–3313 | bootstrap (`join`/`subscribe`) + asynchronous master-master writes: non-conflicting, conflicting and over a stream (`begin`/`commit`/`rollback`) |
+| `tarantool-replication-sync.pcap` | same cluster, node 1 promoted | synchronous replication: `raft_promote` + quorum-acknowledged commits (`raft_confirm`) into a synchronous space |
+
+`tests/run.sh` decodes each fixture with `dist/tarantool.dissector.lua` and
+asserts, per version: zero Lua errors; the expected request types; decoded
+request bodies (e.g. `box.dostring`, `myfunc(2, 3)`, the SQL text); response
+decoding (legacy return codes, SQL `row_count`/`metadata`, the structured error
+stack `[1] ClientError (code 3)` vs. the 1.10 string error); header fields; and
+the 3.x MsgPack ext values (decimal, uuid, datetime, interval). It needs `tshark`
+on `PATH` (override with `TSHARK`; run as a non-root user — Wireshark disables
+`-X lua_script` under root). CI runs it on every change
+(`.github/workflows/tests.yml`) across **Wireshark 3.x and 4.x** — both on
+`ubuntu-24.04`, the 3.x leg pulling Wireshark 3.6 from jammy's packages. A second
+workflow (`dist.yml`) regenerates `dist/` from `src/` and fails if they differ,
+so the committed builds can't drift from the sources.
+
+```sh
+sh tests/run.sh
+# macOS:
+TSHARK=/Applications/Wireshark.app/Contents/MacOS/tshark sh tests/run.sh
+```
+
+### Generating traffic by hand
+
+`test.lua` drives a local Tarantool 3.x instance through most IProto commands —
+CRUD, `call`/`eval`, SQL (`execute`/`prepare`), event watchers, interactive
+transactions over streams and replication (`join`/`subscribe`). Capture loopback
+on port 3301 (e.g. in Wireshark on `lo`/`lo0`) and run `tarantool test.lua`.