UnobservedTaskException from orphaned ReadLineAsync task in Watcher<T>.CreateWatchEventEnumerator during cancellation

[JamesNK]

Description

When a CancellationToken is cancelled during a watch operation (e.g., during application shutdown), the AttachCancellationToken helper inside Watcher<T>.CreateWatchEventEnumerator creates an orphaned ReadLineAsync() task whose IOException is never observed. This causes a TaskScheduler.UnobservedTaskException during GC finalization.

Affected Version

KubernetesClient 19.0.2 (and likely earlier versions using the same CreateWatchEventEnumerator code path)

Root Cause

The bug is in the AttachCancellationToken local function inside CreateWatchEventEnumerator (Watcher.cs):

Task<TR> AttachCancellationToken<TR>(Task<TR> task)
{
    if (!task.IsCompleted)
    {
        // here to pass cancellationToken into task
        return task.ContinueWith(t => t.GetAwaiter().GetResult(), cancellationToken);
    }
    return task;
}

When used with ReadLineAsync():

var line = await AttachCancellationToken(streamReader.ReadLineAsync())
    .ConfigureAwait(false);

The following sequence causes the bug:

1. streamReader.ReadLineAsync() starts reading from the HTTP/2 stream, returning taskA (the original).
2. AttachCancellationToken sees taskA is not completed, so it creates taskB via taskA.ContinueWith(t => t.GetAwaiter().GetResult(), cancellationToken).
3. CreateWatchEventEnumerator awaits taskB.
4. When cancellationToken is cancelled:
   • taskB transitions to Canceled immediately because ContinueWith uses the token to cancel the continuation scheduling, not the original task.
   • await taskB throws OperationCanceledException — normal shutdown path.
   • taskA (the original ReadLineAsync()) is still running in the background.
5. The underlying HTTP/2 connection is torn down (stream disposed, request aborted).
6. taskA faults with IOException("The request was aborted.").
7. Nobody observes taskA's exception — the ContinueWith continuation was already cancelled and never ran, so t.GetAwaiter().GetResult() never executes.
8. When the GC finalizes taskA, .NET raises TaskScheduler.UnobservedTaskException.

Why onError doesn't help

The onError callback inside CreateWatchEventEnumerator only handles k8s Status ERROR events and JSON deserialization exceptions. It is never invoked for transport-level IOException from ReadLineAsync(). Those exceptions would need to propagate out through await AttachCancellationToken(...), but since taskB is cancelled before taskA faults, the IOException never reaches any error handler.

Similarly, in WatcherLoop, the catch (Exception e) { OnError?.Invoke(e); } would handle IOException if it propagated — but OperationCanceledException from the cancelled continuation is caught first, and the method exits before taskA faults.

Reproduction

This occurs in any scenario where:

1. A watch stream is active with WatchAsync<T, L> or through Watcher<T>
2. The CancellationToken is cancelled while ReadLineAsync() is in progress
3. The underlying HTTP connection is subsequently torn down

This is a common pattern during application shutdown. We observed this in .NET Aspire where the k8s client is used for DCP (Developer Control Plane) resource watching. See microsoft/aspire#18388 for the original report with full stack traces.

Stack trace from the UnobservedTaskException:

System.IO.IOException: The request was aborted.
   at System.Net.Http.Http2Connection.Http2Stream.CheckResponseBodyState()
   at System.Net.Http.Http2Connection.Http2Stream.ReadDataAsync(Memory`1 buffer, HttpResponseMessage responseMessage, CancellationToken cancellationToken)
   at System.Net.Http.Http2Connection.Http2Stream.CopyToStream.<ReadAsync>g__WaitForData|1_0(Http2Stream stream, HttpResponseMessage response, Memory`1 buffer, CancellationToken cancellationToken)
   at System.IO.StreamReader.ReadBufferAsync(CancellationToken cancellationToken)
   at System.IO.StreamReader.ReadLineAsyncInternal(CancellationToken cancellationToken)

Suggested Fix

The simplest fix is to observe taskA's exception when the continuation is cancelled. For example:

Task<TR> AttachCancellationToken<TR>(Task<TR> task)
{
    if (!task.IsCompleted)
    {
        // Observe any exception from the original task to prevent UnobservedTaskException
        // when the continuation is cancelled before the original task faults.
        task.ContinueWith(
            static t => { _ = t.Exception; },
            TaskContinuationOptions.OnlyOnFaulted);

        return task.ContinueWith(t => t.GetAwaiter().GetResult(), cancellationToken);
    }
    return task;
}

Alternatively, the pattern could be restructured to use Task.WhenAny or a TaskCompletionSource that properly observes both the original task's result and the cancellation, ensuring no task goes unobserved.

Another option would be to use CancellationTokenSource.CreateLinkedTokenSource and pass the linked token directly to ReadLineAsync() (since StreamReader.ReadLineAsync(CancellationToken) is available on .NET 7+), avoiding the need for AttachCancellationToken entirely.

[tg123]

i prefer net7+ version, this is an old tricky workaround.

[JamesNK]

Targets are net8.0;net9.0;net10.0 so you could make the cleaner fix.