Reduce reader/writer contention in Pipelines

[VSadov]

The change deals with excessive contentions coming from Pipelines in some benchmarks when reader and writer contend for the lock used to protect shared state of the Pipe.

Changes:

• move some operations that may take nontrivial time while not requiring locking outside of the lock. That is mostly renting/returning byte buffers. That could end up in allocations or coordinating with other threads accessing the pool, but by itself does not need to lock the Pipe.
• make the internal per-Pipe pool of segment objects a FIFO, so that writer (which returns segments) and reader (which takes then out) operate at different ends of the pool - to reduce cache line sharing in rent/return operations.
• switch the Pipe lock from monitor lock to Threading.Lock. This lock often inflates anyways.
• schedule continuations to local ThreadPool queues (vs. global) - to keep logical "pipelining" running on the same physical thread, if possible.
  For example, writer thread, if it looks for more work after writing, would be preferred to take care of asynchronous reading and processing of written data, thus further reducing contention and improving locality of access.

=== effect on JSON asp.net benchmark running on a 56-core machine:

• Contentions:

- Max Lock Contention (#/s)               | 483
+ Max Lock Contention (#/s)               | 66

• RPS:

- Requests/sec              | 2,094,374
+ Requests/sec              | 2,148,465

• Throughput:

- Read throughput (MB/s)    | 291.61
+ Read throughput (MB/s)    | 299.14

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[VSadov]

=== typical counts before the PR:

| application                             |                                         |
| --------------------------------------- | --------------------------------------- |
| Max Process CPU Usage (%)               | 87                                      |
| Max Cores usage (%)                     | 4,892                                   |
| Max Working Set (MB)                    | 95                                      |
| Max Private Memory (MB)                 | 710                                     |
| Build Time (ms)                         | 6,801                                   |
| Start Time (ms)                         | 218                                     |
| Published Size (KB)                     | 111,948                                 |
| Symbols Size (KB)                       | 55                                      |
| .NET Core SDK Version                   | 11.0.100-preview.7.26365.114            |
| ASP.NET Core Version                    | 11.0.0-preview.7.26365.114+8c1bd18b8c65 |
| .NET Runtime Version                    | 11.0.0-preview.7.26365.114+8c1bd18b8c65 |
| Max Global CPU Usage (%)                | 92                                      |
| Max CPU Usage (%)                       | 90                                      |
| Max Working Set (MB)                    | 99                                      |
| Max GC Heap Size (MB)                   | 14                                      |
| Size of committed memory by the GC (MB) | 11                                      |
| Max Number of Gen 0 GCs / sec           | 2.00                                    |
| Max Number of Gen 1 GCs / sec           | 1.00                                    |
| Max Number of Gen 2 GCs / sec           | 1.00                                    |
| Max Gen 0 GC Budget (MB)                | 19                                      |
| Max Time in GC (%)                      | 13.00                                   |
| Max Gen 0 Size (B)                      | 0                                       |
| Max Gen 1 Size (B)                      | 1,530,184                               |
| Max Gen 2 Size (B)                      | 1,704,952                               |
| Max LOH Size (B)                        | 0                                       |
| Max POH Size (B)                        | 3,147,488                               |
| Max Allocation Rate (B/sec)             | 7,875,256                               |
| Max GC Heap Fragmentation (%)           | 569%                                    |
| # of Assemblies Loaded                  | 63                                      |
| Max Exceptions (#/s)                    | 896                                     |
| Max Lock Contention (#/s)               | 483                                     |
| Max ThreadPool Threads Count            | 63                                      |
| Max ThreadPool Queue Length             | 131                                     |
| Max ThreadPool Items (#/s)              | 2,510,219                               |
| Max Active Timers                       | 1                                       |
| IL Jitted (B)                           | 231,701                                 |
| Methods Jitted                          | 2,560                                   |


| load                      |            |
| ------------------------- | ---------- |
| Max Process CPU Usage (%) | 47         |
| Max Cores usage (%)       | 2,618      |
| Max Working Set (MB)      | 47         |
| Max Private Memory (MB)   | 363        |
| Start Time (ms)           | 1          |
| Max Global CPU Usage (%)  | 60         |
| First Request (ms)        | 131        |
| Requests/sec              | 2,094,374  |
| Requests                  | 31,623,276 |
| Mean latency (ms)         | 0.24       |
| Max latency (ms)          | 9.13       |
| Bad responses             | 0          |
| Socket errors             | 0          |
| Read throughput (MB/s)    | 291.61     |
| Latency 50th (ms)         | 0.22       |
| Latency 75th (ms)         | 0.28       |
| Latency 90th (ms)         | 0.37       |
| Latency 99th (ms)         | 0.57       |

[VSadov]

=== typical counts after the PR:

| application                             |                                         |
| --------------------------------------- | --------------------------------------- |
| Max Process CPU Usage (%)               | 89                                      |
| Max Cores usage (%)                     | 4,969                                   |
| Max Working Set (MB)                    | 94                                      |
| Max Private Memory (MB)                 | 702                                     |
| Build Time (ms)                         | 8,201                                   |
| Start Time (ms)                         | 207                                     |
| Published Size (KB)                     | 111,948                                 |
| Symbols Size (KB)                       | 55                                      |
| .NET Core SDK Version                   | 11.0.100-preview.7.26365.114            |
| ASP.NET Core Version                    | 11.0.0-preview.7.26365.114+8c1bd18b8c65 |
| .NET Runtime Version                    | 11.0.0-preview.7.26365.114+8c1bd18b8c65 |
| Max Global CPU Usage (%)                | 88                                      |
| Max CPU Usage (%)                       | 85                                      |
| Max Working Set (MB)                    | 98                                      |
| Max GC Heap Size (MB)                   | 15                                      |
| Size of committed memory by the GC (MB) | 12                                      |
| Max Number of Gen 0 GCs / sec           | 3.00                                    |
| Max Number of Gen 1 GCs / sec           | 2.00                                    |
| Max Number of Gen 2 GCs / sec           | 1.00                                    |
| Max Gen 0 GC Budget (MB)                | 36                                      |
| Max Time in GC (%)                      | 85.00                                   |
| Max Gen 0 Size (B)                      | 0                                       |
| Max Gen 1 Size (B)                      | 20,368                                  |
| Max Gen 2 Size (B)                      | 4,378,240                               |
| Max LOH Size (B)                        | 0                                       |
| Max POH Size (B)                        | 3,147,488                               |
| Max Allocation Rate (B/sec)             | 8,455,664                               |
| Max GC Heap Fragmentation (%)           | 717%                                    |
| # of Assemblies Loaded                  | 63                                      |
| Max Exceptions (#/s)                    | 916                                     |
| Max Lock Contention (#/s)               | 66                                      |
| Max ThreadPool Threads Count            | 62                                      |
| Max ThreadPool Queue Length             | 223                                     |
| Max ThreadPool Items (#/s)              | 2,881,024                               |
| Max Active Timers                       | 1                                       |
| IL Jitted (B)                           | 240,531                                 |
| Methods Jitted                          | 2,791                                   |


| load                      |            |
| ------------------------- | ---------- |
| Max Process CPU Usage (%) | 48         |
| Max Cores usage (%)       | 2,669      |
| Max Working Set (MB)      | 47         |
| Max Private Memory (MB)   | 363        |
| Start Time (ms)           | 1          |
| Max Global CPU Usage (%)  | 61         |
| First Request (ms)        | 141        |
| Requests/sec              | 2,148,465  |
| Requests                  | 32,441,366 |
| Mean latency (ms)         | 0.23       |
| Max latency (ms)          | 5.70       |
| Bad responses             | 0          |
| Socket errors             | 0          |
| Read throughput (MB/s)    | 299.14     |
| Latency 50th (ms)         | 0.21       |
| Latency 75th (ms)         | 0.28       |
| Latency 90th (ms)         | 0.37       |
| Latency 99th (ms)         | 0.56       |

[Copilot]

Pull request overview

This PR updates System.IO.Pipelines internals to reduce reader/writer lock contention by changing how BufferSegment instances and backing memory are acquired/returned, and adjusts related scheduling and tests.

Changes:

• Switch Pipe’s segment pooling from a stack to an SPSC queue and return consumed segments outside the main lock.
• Speculatively rent backing memory outside the lock and add a lock-free fast path for Advance when writing is active.
• Update the segment-pool reuse test expectation (FIFO) and set ThreadPool scheduling to prefer local queues.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/libraries/System.IO.Pipelines/tests/BufferSegmentPoolTest.cs	Updates pooling test to expect FIFO reuse order.
src/libraries/System.IO.Pipelines/src/System/IO/Pipelines/ThreadPoolScheduler.netcoreapp.cs	Uses preferLocal: true for ThreadPool scheduling.
src/libraries/System.IO.Pipelines/src/System/IO/Pipelines/PipeOptions.cs	Updates segment pool sizing defaults/comments (but currently misses required property assignment).
src/libraries/System.IO.Pipelines/src/System/IO/Pipelines/Pipe.cs	Moves to SPSC pooling + reduces time under the shared lock (memory rent + segment return).
src/libraries/System.IO.Pipelines/src/System.IO.Pipelines.csproj	Links in SPSC queue + padding shared sources (and introduces a BOM).

[VSadov]

Benchmark used (--scenario json --profile aspnet-gold-lin):

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/platform.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/azure.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/azure.profile.yml --scenario json --profile aspnet-gold-lin --load.connections 512 --application.framework net11.0 --application.collectDependencies true --application.options.collectCounters true --load.options.reuseBuild true  --application.buildArguments "-p:Features=runtime-async=on"

The changes to notice:

• Contentions:

- Max Lock Contention (#/s)               | 483
+ Max Lock Contention (#/s)               | 66

• RPS:

- Requests/sec              | 2,094,374
+ Requests/sec              | 2,148,465

• Throughput:

- Read throughput (MB/s)    | 291.61
+ Read throughput (MB/s)    | 299.14

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[VSadov]

Safe enough for us. Overestimating due to concurrent dequeuing should be rare and just means that an edge case when we cannot recycle a segment due to queue being too full may happen, while if we had up to date count it would not. (or it would still happen if enqueuer was a bit faster).

The Count is O(n) only in transitional state when queue grows to larger capacity. Otherwise there is only one underlying segment.
Because of double-up growth policy, even in edge cases, the queue can grow only a few times before reaching limits of our use.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

[VSadov]

this is not a real "leak". It is ok to GC in exceptional situations.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[VSadov]

The diff looks big, but it is the same code as before just wrapped in a try and the segment-releasing loop (while (returnStart != null && returnStart != returnEnd)) is taken out of lock and into the finally that follows.

[VSadov]

Build failures on NET481 seems unrelated to the change:

D:\a_work\1\s\src\libraries\Microsoft.Extensions.Caching.Abstractions\tests\TrimmingTests\HybridCacheEntryOptionsAccessorTest.cs(17,6): error CS0246: The type or namespace name 'UnsafeAccessorAttribute' could not be found (are you missing a using directive or an assembly reference?) [D:\a_work\1\s\src\libraries\Microsoft.Extensions.Caching.Abstractions\tests\Microsoft.Extensions.Caching.Abstractions.Tests.csproj::TargetFramework=net481]

#130892