[🚀 Feature]: API suggestions For BIDI

[NMSAzulXXiaoHao]

Description

I've been using Selenium .NET and have encapsulated a set of simple and practical APIs. However, I've only recently discovered the BIDI technology. I've encountered some troubles when trying to encapsulate it. So far, it doesn't fully match the CDP, and in some functions, it can't replace the CDP. I want to express my needs from the developer's perspective, hoping it can be helpful to the development of the BIDI technology.

Have you considered any alternatives or workarounds?

The following is the basic logical implementation. In fact, I value various methods in BindInterceptRequestAsync more. At least, it won't have strange semantic calls like req.Request.Request. Also, for GetDataAsync and ContinueWithNewBody, there is no need to worry about encoding issues and recombining Response headers, etc. Regarding data from Websocket, streaming SSE, etc., I think handling long - connection requests is a unique type of requirement. At least, their protocols are not completely the same. Before, I used RequestId to determine whether these requests came from the same connection. This is not only a burden for development and maintenance, but also a performance problem in high - frequency and high - throughput testing scenarios.

BiDi Network - Enhanced Methods

Register User Handler

// IUserInterceptors -  User Handler
// Users need to implement the abstract class IUserInterceptors and pass it to BiDi.
// The IUserInterceptors class should have methods to determine if a request matches certain criteria (e.g., URL patterns ,response body) and a handler method to modify the request or response body.
// For example, the IUserInterceptors class could look like this:
public abstract class IUserInterceptors
{
     public virtual bool IsMatchRequest(RequestInfo request)){return true;}
     public virtual bool IsMatchBody(string body)){return true;}
     public abstract Task<string> HandlerAsync(string body);
}

Registe Handler

// Then, users can create their own interceptors by inheriting from IUserInterceptors and implementing the methods to define their interception logic.
// Finally, we can add the user's interceptors to the Bidi handler and set up the interception logic in the Network domain.
// For example:
public bool TryAddInterceptAsync(new MyInterceptors());

Bind Intercept Event

private async Task BindInterceptRequestAsync()
{
    await InterceptRequestAsync(async req =>
    {
        // First, we can get the original request or response body using the GetDataAsync method.
        // This allows us to analyze the content and determine if it matches the user's interception criteria.
        // For example, we can get the response body for a request like this:
        // This method automatically decodes the Base64 ciphertext.
        //  if req.Response.BodyEncryption == KnownHttpEncryption.Base64 then return Encoding.UTF8.GetString(Convert.FromBase64String(orginalBody)) else return orginalBody
        // Note:
        //  We should be careful when handling large response bodies, as it may consume a lot of memory.
        //  We can consider adding some limits or streaming the data if necessary.
        string body = await GetDataAsync(DataType.Response, req);

        foreach (var interceptor in UserInterceptorCollection)
        {
            // We can choose to match the request type (e.g., only intercept scripts or XHR requests) or match specific URLs or patterns.
            // For example, if we only want to intercept JavaScript files, we can check the request's resource type:
            // IsMatchRequest(RequestInfo request){ return request.KnownResourceType == KnownHttpResourceType.JSScript && request.Url.Contains("core.js"); }
            // We can also choose to match the request body (e.g., only intercept responses that contain certain keywords or patterns).
            // For example, if we only want to intercept responses that contain a specific function call, we can check the response body:
            // IsMatchBody(string body){ return body.Contains("myFunctionCall"); }
            if (interceptor.IsMatchRequest(req) && interceptor.IsMatchBody(body))
            {

                // If the request matches the interceptor's criteria, we can call the handler method to modify the request or response body.
                body = await interceptor.HandlerAsync(body);

                // After modifying the body, we can provide the modified response back to the browser using the ProvideResponseAsync method.
                // We can provide the modified response like this:
                // await req.ContinueWithNewBody(body);return;
                // Note: We need to encode the modified body back to Base64 if it was originally Base64 encoded.
                // For example, if the original body was Base64 encoded, we can encode the modified body like this:
                // ContinueWithNewBody(string/ReadOnlySpan<char> body){ if req.Response.BodyEncryption == KnownHttpEncryption.Base64 then return.....};
                // No matter Header or Body, we can modify and provide the new response using ProvideResponseAsync, but if we only want to modify the Body,
                // we can use ContinueWithNewBody for better performance.

                // Note: If we use ContinueWithReponseAsync(), we need to provide the entire response including headers and body,
                // but if we only want to modify the body, we can use ContinueWithNewBody which is more efficient.
                await req.ContinueWithNewBodyAsync(body);
                return;
            }
        }
        // If the request does not match any interceptor's criteria, we can simply continue with the original request or response without modification.
        // For example, we can continue with the original request like this:
        await req.ContinueAsync();
    });
}

[selenium-ci]

@NMSAzulXXiaoHao, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[nvborisenko]

Thanks for your interest, and suggestions.

Currently we are focusing on low-level implementation. If low-level works good, then any high-level works good too implicitly. Some "helpers" left when we just started the implementation, we are removing them (eg #17288). Everything is for just finalizing Public API: #16095. But we are trying to design this API to be extensible in mind, so anybody is free to create everything like you created.

So, together we can make low-level better. And performant. What real problems you face?

PS: I would recommend to use Nighly Builds - not yet released significantly better performance especially in your network interception topic.

[Chandan25sharma]

I was digging into how the bindings are structured and wanted to suggest a couple of ways this could be added to make life easier for developers:

1. High-Level Extension Methods
   Since provideResponse in the W3C spec requires the entire response to be replaced, we could add some extension methods (or helpers) to InterceptedRequest / InterceptedResponse in NetworkModule.HighLevel.cs to hide the boilerplate.

For example, fetching the string and passing through the original headers could look like this:

csharp
// Helper to automatically handle the Base64/String decoding
public async Task GetResponseBodyAsStringAsync(CancellationToken cancellationToken = default)
{
var bytesData = await BiDi.Network.GetDataAsync(DataType.Response, Request.Request, null, cancellationToken);
return (string)bytesData; // BytesValue already handles the explicit operator casting
}
// Helper to keep original headers/status code, while ONLY safely swapping the body
public Task ContinueWithNewBodyAsync(string newBody, CancellationToken cancellationToken = default)
{
var options = new ProvideResponseOptions
{
Body = new StringBytesValue(newBody),
StatusCode = Response.Status,
Headers = Response.Headers // Automatically passes through the original headers behind the scenes
};
return BiDi.Network.ProvideResponseAsync(Request.Request, options, cancellationToken);
}

The INetwork Abstraction

In the long term, I noticed the Selenium bindings already have an INetwork abstraction (NetworkManager) with a NetworkRequestHandler and NetworkResponseHandler. It currently uses CDP, but once the maintainers port it to back onto BiDi, we should natively get the exact IsMatchRequest and handler pattern you described without needing to write custom interceptor wrappers!

csharp
// Once NetworkManager uses BiDi natively, this CDP pattern will just work:
var handler = new NetworkResponseHandler
{
ResponseMatcher = response => response.Url.Contains("core.js"),
ResponseTransformer = response =>
{
response.Body = response.Body.Replace("oldCode", "newCode");
return response;
}
};