Improve network throughput#20
Open
Sobottasgithub wants to merge 65 commits into
Open
Conversation
Flake lock file updates:
• Updated input 'ttp2':
'github:Sobottasgithub/ttp2/352b32ee3cec2e29a054597d90353adea87c058b?narHash=sha256-HJOsVaFAIrsZanayn3ZWY0fOdQ0SYaLO9i7wFElioO8%3D' (2026-04-26)
→ 'github:Sobottasgithub/ttp2/d1a3b10474adb065395b7f508ec953e774d76a13?narHash=sha256-PTnhNzLTtT/BKjUlFbkZ/Vo1F/6e5SqbkFMATD/CzBQ%3D' (2026-05-14)
Link Tablo against TTP2
Redesign flake outputs
Collaborator
|
May the AI review first |
There was a problem hiding this comment.
Pull request overview
This PR migrates Tablo’s networking stack away from the in-repo tabnet protocol/UDP discovery implementation to the external TTP2 (transfer protocol + session controllers) and TUD (UDP discovery) libraries, aiming to improve throughput (including epoll-based accept loops). It also updates the client CLI to support sending CSV files and requesting viewports, and refactors build packaging (CMake options + Nix flake inputs).
Changes:
- Replace legacy
tabnetprotocol + UDP discovery + session controllers with TTP2/TUD-based session management across Client/Master/Node. - Introduce epoll-based connection handling in new
NetworkManagerimplementations for Master and Node. - Update build/deploy tooling: CMake build options, Nix flake inputs/packaging, and README instructions; remove protobuf-based tabnet code + TCTest.
Reviewed changes
Copilot reviewed 46 out of 47 changed files in this pull request and generated 19 comments.
Show a summary per file
| File | Description |
|---|---|
| TCTest/src/tabcrypt_test.h | Removed tabcrypt test header. |
| TCTest/src/main.cpp | Removed tabcrypt test executable. |
| TCTest/CMakeLists.txt | Removed TCTest build target. |
| TabloNode/src/utils/worker.h | Switch worker API from tabnet packets to TTP2 packet types; add CSV manager integration. |
| TabloNode/src/utils/worker.cpp | Update request/response processing to use TTP2 payload variants and CSV viewport generation. |
| TabloNode/src/utils/udp_discovery.h | Removed legacy node UDP discovery. |
| TabloNode/src/utils/udp_discovery.cpp | Removed legacy node UDP discovery implementation. |
| TabloNode/src/utils/networking.cpp | Removed legacy node TCP networking implementation. |
| TabloNode/src/utils/network_manager.h | Rename/adjust header guard + class naming to NetworkManager. |
| TabloNode/src/utils/network_manager.cpp | New epoll-based node connection accept loop + TUD discovery + TTP2 session wiring. |
| TabloNode/src/utils/csv_manager.h | New CSV helper abstraction for file + viewport operations. |
| TabloNode/src/utils/csv_manager.cpp | New CSV parsing/slicing helpers used by node worker. |
| TabloNode/src/tabloNode.cpp | Update node entrypoint to use NetworkManager. |
| TabloNode/CMakeLists.txt | Link node against new libraries ttp2 and tud. |
| TabloMaster/src/utils/udp_discovery.h | Removed legacy master UDP discovery. |
| TabloMaster/src/utils/udp_discovery.cpp | Removed legacy master UDP discovery implementation. |
| TabloMaster/src/utils/node_session_controller.h | Removed legacy node session controller (tabnet-based). |
| TabloMaster/src/utils/node_session_controller.cpp | Removed legacy node session controller implementation. |
| TabloMaster/src/utils/networking.h | Removed legacy master networking orchestrator. |
| TabloMaster/src/utils/networking.cpp | Removed legacy master networking implementation. |
| TabloMaster/src/utils/network_manager.h | New master NetworkManager interface for TTP2/TUD-based orchestration. |
| TabloMaster/src/utils/network_manager.cpp | New epoll-based accept loop + TUD server discovery + node fan-out using TTP2 controllers. |
| TabloMaster/src/utils/client_session_controller.h | Removed legacy client session controller (tabnet-based). |
| TabloMaster/src/utils/client_session_controller.cpp | Removed legacy client session controller implementation. |
| TabloMaster/src/tabloMaster.cpp | Update master entrypoint to use NetworkManager. |
| TabloMaster/CMakeLists.txt | Link master against new libraries ttp2 and tud. |
| TabloClient/src/utils/networking.h | Removed legacy client networking wrapper. |
| TabloClient/src/utils/networking.cpp | Removed legacy client networking implementation. |
| TabloClient/src/utils/network_manager.h | New client network manager wrapper around TTP2 client session controller. |
| TabloClient/src/utils/network_manager.cpp | Implement non-blocking connect + background TTP2 networking session. |
| TabloClient/src/utils/cli.h | Update CLI to take parsed argv struct; add sendFile helper. |
| TabloClient/src/utils/cli.cpp | Add interactive options to send packets, send CSV file, and request viewport; read TTP2 responses. |
| TabloClient/src/utils/argv_struct.h | New struct for parsed CLI args. |
| TabloClient/src/tabloClient.cpp | Parse --master and --file args and construct CLI with Argv. |
| TabloClient/CMakeLists.txt | Link client against ttp2. |
| README.md | Update Nix build/run commands and link to TTP2/TUD repos. |
| lib/libtabnet/src/tabnet.cpp | Remove tabnet implementation (now empty namespace). |
| lib/libtabnet/src/protobuf/transfer_protocol.proto | Removed protobuf schema. |
| lib/libtabnet/src/protobuf/transfer_protocol.pb.h | Removed generated protobuf header. |
| lib/libtabnet/src/protobuf/transfer_protocol.pb.cc | Removed generated protobuf source. |
| lib/libtabnet/include/tabnet.h | Remove legacy API surface (now empty namespace). |
| lib/libtabnet/include/methods.h | Removed legacy method enum. |
| lib/libtabnet/CMakeLists.txt | Remove protobuf generation/linking; build a minimal shared library. |
| flake.nix | Add TTP2/TUD flake inputs and restructure packages (mkTabloPackage + symlink join). |
| flake.lock | Lock TTP2/TUD inputs. |
| CMakeLists.txt | Add build options for libs/executables and gate subdirectories accordingly. |
| build-proto.sh | Removed protobuf build helper script. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+17
to
+20
| bool hasResponse(); | ||
| ttp2::ClientSessionController::Packet popResponse(); | ||
| void pushRequest(ttp2::ClientSessionController::Packet packet); | ||
|
|
Comment on lines
+3
to
+10
| #include <client_session_controller.h> | ||
| #include <iostream> | ||
| #include <netinet/in.h> | ||
| #include <sys/socket.h> | ||
| #include <arpa/inet.h> | ||
| #include <thread> | ||
| #include <memory> | ||
| #include <type_traits> |
Comment on lines
+8
to
+17
| #include <arpa/inet.h> | ||
| #include <bits/stdc++.h> | ||
| #include <ctime> | ||
| #include <iostream> | ||
| #include <netinet/in.h> | ||
| #include <sys/socket.h> | ||
| #include <system_error> | ||
| #include <thread> | ||
| #include <vector> | ||
| #include <memory> |
Comment on lines
+47
to
+50
| struct epoll_event serverEvents; | ||
| serverEvents.events = EPOLLIN | EPOLLOUT; | ||
| serverEvents.data.fd = serverSocket; | ||
| if (epoll_ctl(epollFd, EPOLL_CTL_ADD, serverSocket, &serverEvents) == -1) { |
Comment on lines
+64
to
+70
| int clientSocket = accept4(serverSocket, nullptr, nullptr, SOCK_NONBLOCK); | ||
| std::wcout << "New clientSocket: " << clientSocket << std::endl; | ||
|
|
||
| clientConnections.push_back(std::thread([this, serverSocket, clientSocket]() { | ||
| this->handleClientConnection(serverSocket, clientSocket); | ||
| })); | ||
| } |
Comment on lines
+18
to
+24
| int CsvManager::getRowCount() { | ||
| int count = 0; | ||
| for (int index = 0; index < this->file.payload.length(); index++) | ||
| if (this->file.payload[index] == '\n') | ||
| count++; | ||
| return count; | ||
| } |
Comment on lines
+51
to
+57
| if (index == 1) { | ||
| for (int countIndex = 0; countIndex < this->file.payload.length(); countIndex++) { | ||
| if (this->file.payload[countIndex] == '\n') | ||
| return resultRow; | ||
| resultRow = resultRow + this->file.payload[countIndex]; | ||
| } | ||
| } |
Comment on lines
9
to
13
| if (argc > 0) { | ||
| for(int index = 0; index < argc; index++) { | ||
| if (std::string(argv[index]).rfind("--interface", 0) == 0) { | ||
| Networking networking(argv[index+1]); | ||
| NetworkManager networkManager(argv[index+1]); | ||
| } |
Comment on lines
10
to
15
| if (argc > 0) { | ||
| for(int index = 0; index < argc; index++) { | ||
| if (std::string(argv[index]).rfind("--interface", 0) == 0) { | ||
| Networking networking(argv[index+1]); | ||
| NetworkManager networkManager(argv[index+1]); | ||
| } | ||
| } |
Comment on lines
10
to
18
| if (argc > 0) { | ||
| struct Argv commandLineArguments; | ||
|
|
||
| for(int index = 0; index < argc; index++) { | ||
| if (std::string(argv[index]).rfind("--master", 0) == 0) { | ||
| Cli cli(argv[index+1]); | ||
| commandLineArguments.tabloMaster = argv[index+1]; | ||
| } else if (std::string(argv[index]).rfind("--file", 0) == 0) { | ||
| commandLineArguments.filePath = argv[index+1]; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add TTP2
Visit the own repository of the Tablo Transfer Protocol
features:
Add TUD
Visit the own repository of Tablo Udp Discovery
Improved Cli: