[POC] Virtual Threads

[robfrank]

This pull request introduces support for using Java virtual threads (Project Loom) across several components of ArcadeDB to improve scalability and concurrency, especially for query execution and network protocols. It adds new configuration options to enable or disable virtual threads for different subsystems, refactors executors to support this feature, and adds tests to verify correct behavior with both virtual and platform threads.

Key changes include:

Virtual Thread Support and Configuration:

• Added new configuration options in GlobalConfiguration.java to enable virtual threads for query engines, HTTP server, HA replication, PostgreSQL, and Redis protocols. These are enabled by default and allow users to toggle between virtual and platform threads as needed. [1] [2] [3] [4]

Query Engine Refactoring:

• Refactored JavaQueryEngine and PolyglotQueryEngine to use an ExecutorService that can be either a virtual thread executor or a fixed thread pool, based on the new configuration. This improves scalability for concurrent query execution. [1] [2] [3] [4] [5]

Network Protocols and Server Integration:

• Updated the gRPC server (GrpcServerPlugin.java) to use a virtual thread executor for handling requests, enhancing its ability to manage many concurrent connections. [1] [2] [3]
• Modified the PostgreSQL protocol listener (PostgresNetworkListener.java) to conditionally use virtual threads for each connection, controlled by the new configuration.

Testing:

• Added comprehensive tests for virtual thread support in both the Java query engine and PostgreSQL protocol, ensuring correct configuration, concurrent execution, and compatibility with both thread types. [1] [2]

Minor and Related Improvements:

• Improved logging and metrics reporting in performance test classes, and clarified comments and parameter names for better maintainability. [1] [2] [3] [4] [5]

These changes collectively make ArcadeDB more scalable and ready for high-concurrency workloads by leveraging modern Java threading capabilities.

[mergify]

🧪 CI Insights

Here's what we observed from your CI run for d3f490d.

🟢 All jobs passed!

But CI Insights is watching 👀

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ -8.24%	✅ 61.11%

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (9c66952)	70174	46242	65.90%
Head commit (d3f490d)	70228 (+54)	40488 (-5754)	57.65% (-8.24%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#2778)	72	44	61.11%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences