Be more tolerant of forbidden headers

[itowlson]

When I write middleware-style components that pass an incoming request on to an upstream, I use into_request() to turn them into WASI requests.

This always fails because the incoming request has Connection and Host headers which are FORBIDDEN. I have to remove those manually to be able to into_request() it. E.g. https://github.com/spinframework/http-auth-middleware/blob/main/github-oauth/src/api/authenticate.rs#L35C1-L37C54. I don't know if other headers are FORBIDDEN but I live in fear that one day a varlet will send me a request that contains such a header and my middleware will crash because I didn't know to remove it.

How terrible would it be to make into_request() quietly ignore forbidden headers, or not forbid them and instead let WASI get mad if it needs to, or in some other way make header-related-forbiddances not the app developer's problem?

Or should we make it so that these headers are not forbidden when used in the middleware/composition scenario? Really I'd expect that headers like these should be passed through the composition chain without anyone caring.

[itowlson]

Spin apps normally receive the Host header, so it seems like they should also be able to receive it when behind middleware.

[fibonacci1729]

Unfortunately, this isn't something that the sdk directly controls afaict. According to the docs, it should be possible to construct a request with invalid headers and the responsibility of enforcing forbidden headers is left to handler.handle. However, i'm quite surprised this is happening since the request is happening intra-component.

Not that it helps now, but I'd hope that the latest wasi:http proposal for delineating outgoing request via client.send will mitigate this where only true outgoing requests are subject to forbidden header enforcement.

Nevertheless, I think your question is valid for the near term.

[itowlson]

As far as I can tell, the error happens in into_request, as far as I can tell, before the constructed request object enters send/handle. I am not sure where inside that it's happening - maybe there is something in the host's implementation of the request resource constructor, or in wasi's wrapping, or in http::Request's implementation of the trait? (although the latter sounds unlikely)

[fibonacci1729]

After poking, indeed it does appear to be happening during the conversion in into_request.