Add awakened_from_remote() to bypass remote_ready_queue by Akatsukis · Pull Request #1 · category-labs/fiber

Akatsukis · 2026-04-06T19:59:07Z

Adds a virtual method algorithm::awakened_from_remote() that allows scheduling algorithms with thread-safe ready-queues to handle cross-thread fiber wakeups directly, avoiding the priority inversion caused by remote_ready_queue_ draining only on the owning thread.

Guarded by BOOST_FIBERS_WORKER_QUEUE_LOCK to protect worker_queue_ for cross-thread detach/attach with zero overhead when not needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

These mutex types are unused by the top-level project. Removes source files, headers, and references from CMakeLists.txt and all.hpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Adds a new scheduling extension point to allow certain scheduling algorithms to directly handle cross-thread wakeups, aiming to reduce latency/priority inversion caused by draining remote_ready_queue_ only on the owning thread.

Changes:

Introduces boost::fibers::algo::algorithm::awakened_from_remote() (default false) as an optional fast-path for remote scheduling.
Updates scheduler::schedule_from_remote() to try awakened_from_remote() before falling back to remote_ready_queue_.
Adds an optional BOOST_FIBERS_WORKER_QUEUE_LOCK spinlock to protect worker_queue_ for cross-thread attach/detach and adjusts related code paths.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
`src/scheduler.cpp`	Uses the new `awakened_from_remote()` fast-path and adds optional locking around `worker_queue_` operations.
`include/boost/fiber/scheduler.hpp`	Declares optional `worker_splk_` and enforces multi-thread support when `BOOST_FIBERS_WORKER_QUEUE_LOCK` is enabled.
`include/boost/fiber/algo/algorithm.hpp`	Adds the new virtual `awakened_from_remote()` API and documents intended usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Unused by the top-level project. Removes source, header, examples, and performance benchmarks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Removes header, example, and include references from fiber.hpp and all.hpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Removes source, header, dedicated test files, and dead do_wait() references in fiber tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Removes fss.hpp, detail/fss.hpp, test files, and cleans fss_data_ from context.hpp and context.cpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Removes header, test files, and examples that solely used unbuffered_channel. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unused by the top-level project. Removes rtm.hpp, spinlock_rtm.hpp, and strips BOOST_USE_TSX branches from spinlock.hpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-13T19:30:46Z

+#if defined(BOOST_FIBERS_NO_SLEEP_QUEUE)
+    BOOST_ASSERT_MSG( false, "timed wait not supported with BOOST_FIBERS_NO_SLEEP_QUEUE");
+    return false;
+#else


With BOOST_FIBERS_NO_SLEEP_QUEUE defined, these wait_until() overloads assert and then return false. In release builds (asserts compiled out) this will silently report an immediate timeout, which can break higher-level primitives (e.g., condition_variable_any timed waits) in a hard-to-diagnose way. Consider making this a hard failure even in release (terminate/abort) or removing/ifdef’ing out timed-wait APIs when BOOST_FIBERS_NO_SLEEP_QUEUE is enabled so misuse can’t silently change semantics.

Copilot · 2026-04-13T19:30:46Z

+#if defined(BOOST_FIBERS_NO_SLEEP_QUEUE)
+    BOOST_ASSERT_MSG( false, "timed wait not supported with BOOST_FIBERS_NO_SLEEP_QUEUE");
+    return false;
+#else


Same issue as the other wait_until() overload: under BOOST_FIBERS_NO_SLEEP_QUEUE this will silently behave as an immediate timeout in non-assert builds. Consider enforcing this at compile time or failing hard in all build modes to prevent subtle logic errors in timed-wait callers.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-23T19:37:53Z

+#if defined(BOOST_FIBERS_AWAKENED_FROM_REMOTE)
+    // Let the algorithm handle cross-thread scheduling directly if it
+    // supports a thread-safe ready-queue (e.g. a shared priority queue).
+    // Gated by BOOST_FIBERS_AWAKENED_FROM_REMOTE because this path requires
+    // the atomic sleep_linked_ mirror (so sleep_is_linked() is safe to
+    // read remotely) and the worker_splk_ spinlock (so the subsequent
+    // ctx->detach() inside awakened_from_remote does not race the owning
+    // thread's attach of another fiber).
+    if ( ! ctx->is_context( type::pinned_context) && ! ctx->sleep_is_linked() ) {
+        if ( algo_->awakened_from_remote( ctx) ) {
+            algo_->notify();
+            return;
+        }
+    }


The new cross-thread fast-path (awakened_from_remote() + atomic sleep_linked_ mirror) introduces new concurrency behavior but there are no tests exercising (1) that awakened_from_remote() is invoked for eligible remote wakeups, (2) that sleeping/pinned contexts correctly fall back to remote_ready_queue_, and (3) that no duplicate scheduling occurs under races. Given the existing multi-threaded test suite, it would be good to add a focused regression test with a custom algorithm overriding awakened_from_remote() and counters/assertions for these cases.

Extends algorithm::awakened_from_remote() so algorithms with a thread-safe ready queue (e.g. a shared priority queue) can bypass the per-thread remote_ready_queue_ FIFO on cross-thread wakeups. When defined, BOOST_FIBERS_AWAKENED_FROM_REMOTE also enables: - a spinlock around attach/detach_worker_context, so ctx->detach() from a remote thread (as performed inside awakened_from_remote) does not race the owning thread's attach for a different fiber. - an atomic mirror of the sleep-queue link state on context, so sleep_is_linked() can be queried from a remote thread without touching the non-atomic intrusive sleep_hook_. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

jhunsaker and others added 2 commits March 27, 2026 18:05

Remove unused CUDA and HIP headers from fiber submodule

5127e91

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove timed and recursive mutex support

34bb6b6

These mutex types are unused by the top-level project. Removes source files, headers, and references from CMakeLists.txt and all.hpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Akatsukis mentioned this pull request Apr 6, 2026

[fiber] Add awakened_from_remote() to bypass remote_ready_queue for cross-thread fiber scheduling category-labs/monad#2119

Open

Akatsukis requested a review from Copilot April 6, 2026 20:45

Copilot started reviewing on behalf of Akatsukis April 6, 2026 20:46 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

Comment thread src/scheduler.cpp Outdated

Comment thread src/scheduler.cpp Outdated

Comment thread src/scheduler.cpp Outdated

Comment thread include/boost/fiber/algo/algorithm.hpp

Comment thread include/boost/fiber/algo/algorithm.hpp Outdated

Akatsukis force-pushed the xiaojun/awakened_from_remote branch 2 times, most recently from 08c6e7b to d4627cd Compare April 7, 2026 13:57

jhunsaker and others added 11 commits April 7, 2026 22:50

Remove work_stealing algorithm

8c71e32

Unused by the top-level project. Removes source, header, examples, and performance benchmarks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove shared_work algorithm

5c37e0e

Unused by the top-level project. Removes source, header, examples, and performance benchmarks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove segmented_stack

a58880a

Unused by the top-level project. Removes header, example, and include references from fiber.hpp and all.hpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove fiber barrier

baad893

Unused by the top-level project. Removes source, header, dedicated test files, and dead do_wait() references in fiber tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove pooled_fixedsize_stack

fd59853

Unused by the top-level project. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove fiber-specific storage (fss)

b491046

Unused by the top-level project. Removes fss.hpp, detail/fss.hpp, test files, and cleans fss_data_ from context.hpp and context.cpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove unbuffered_channel

a8c17f4

Unused by the top-level project. Removes header, test files, and examples that solely used unbuffered_channel. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove performance benchmarks

36e5d04

Unused by the top-level project. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove asio examples

dd5c7f3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove RTM (Intel TSX) spinlock support

952434c

Unused by the top-level project. Removes rtm.hpp, spinlock_rtm.hpp, and strips BOOST_USE_TSX branches from spinlock.hpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove unused thread_barrier

24ef24e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Akatsukis force-pushed the xiaojun/awakened_from_remote branch from d4627cd to 7f160f5 Compare April 9, 2026 14:00

Akatsukis requested a review from Copilot April 9, 2026 15:28

Copilot started reviewing on behalf of Akatsukis April 9, 2026 15:29 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

Comment thread include/boost/fiber/scheduler.hpp Outdated

Akatsukis force-pushed the xiaojun/awakened_from_remote branch 2 times, most recently from 84fb830 to dbaa8a9 Compare April 9, 2026 16:54

Akatsukis requested a review from Copilot April 9, 2026 17:04

Copilot started reviewing on behalf of Akatsukis April 9, 2026 17:05 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

Comment thread src/scheduler.cpp Outdated

Akatsukis force-pushed the xiaojun/awakened_from_remote branch 2 times, most recently from 435a0f3 to 595be01 Compare April 13, 2026 19:07

Akatsukis requested a review from Copilot April 13, 2026 19:23

Copilot started reviewing on behalf of Akatsukis April 13, 2026 19:24 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Akatsukis marked this pull request as ready for review April 14, 2026 16:03

Akatsukis force-pushed the xiaojun/awakened_from_remote branch from 595be01 to f8c75be Compare April 23, 2026 17:28

Akatsukis requested a review from Copilot April 23, 2026 19:03

Copilot started reviewing on behalf of Akatsukis April 23, 2026 19:04 View session

Copilot AI reviewed Apr 23, 2026

View reviewed changes

Comment thread include/boost/fiber/scheduler.hpp

Comment thread src/scheduler.cpp Outdated

Comment thread src/scheduler.cpp

Comment thread include/boost/fiber/context.hpp Outdated

Akatsukis force-pushed the xiaojun/awakened_from_remote branch from f8c75be to 04b5548 Compare April 23, 2026 19:28

Akatsukis requested a review from Copilot April 23, 2026 19:30

Copilot started reviewing on behalf of Akatsukis April 23, 2026 19:31 View session

Copilot AI reviewed Apr 23, 2026

View reviewed changes

jhunsaker force-pushed the xiaojun/awakened_from_remote branch from 04b5548 to 49a83fe Compare April 28, 2026 14:11

Akatsukis marked this pull request as draft May 13, 2026 15:42

jhunsaker force-pushed the category branch 2 times, most recently from 133ac78 to a7c0273 Compare May 29, 2026 05:45

jhunsaker force-pushed the category branch from a7c0273 to a0764ba Compare June 11, 2026 22:50

Uh oh!

Conversation

Akatsukis commented Apr 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants