Servlet header rework regresses throughput since 7.0.5

[180254]

Observations on our service overhead tests

I found a problem while conducting "overhead" tests on our service. By "overhead" I mean tests in which the data processing is mocked - it does no actual work and simply returns fake data, but all service architecture blocks are involved.

Upgrading Spring Framework 7.0.4 → 7.0.5 (via Spring Boot 4.0.2 → 4.0.3) caused a ~8x throughput drop on one of our Spring MVC (Servlet, Jetty) services. The problem also occurs on 7.0.8. We bisected it to a single change: reverting org.springframework.http.server.ServletServerHttpRequest and org.springframework.http.server.ServletServerHttpResponse to their 7.0.4 versions (while keeping Spring Framework current (7.0.8) for everything else) fully restores throughput.

A brief overview of our service and how it relates to the problem:

• The service runs on Spring MVC behind Jetty, is served over HTTP/2, and is CPU-constrained (~2 vCPU). It touches HTTP headers many times per request, on both sides of the exchange.
• Request side: every request reads a number of incoming headers (identity/authorization and various metadata supplied by our API management gateway) to make per-request decisions. Some headers are checked multiple times across different subcomponents and filters.
• Response side: responses are produced as ResponseEntity, so the framework pipeline (content negotiation, message conversion, and container-level compression) reads the response headers several times per request. A couple of custom headers are also set early, on the raw servlet response, before serialization.

References

• Optimize request and response header handling in Spring MVC #36334
• Consistent adaptation of HTTP headers on Servlet responses #36343
• Avoid update to request "Content-Type" for characterEncoding in ServletServerHttpRequest #36426

Minimal sample application

https://github.com/180254/spring-framework-issue-37042

What do the "shadowServletServerHttpRequest" and "shadowServletServerHttpResponse" profiles do? These are Maven profiles activated in the sample app. Each profile copies its corresponding class (ServletServerHttpRequest or ServletServerHttpResponse), taken from Spring Framework 7.0.4, onto the classpath, so that it overrides ("shadows") the version otherwise provided. In effect, I'm running a newer version of the Spring Framework but with these specific classes reverted to their 7.0.4 form.

Please see REQ/S for various cases:

$ ./matrix.sh 

===== SUMMARY (req/s; responseHeaderCount=50 responseHeaderReads=50 requestHeaderReads=50, server CPUs=0-1) =====
CONTAINER VERSION                                                                  PROTO    REQ/S      CORES   
--------- ----------------                                                         -----    -----      -----   
tomcat    7.0.4                                                                    h1       2830.94    1.97    
tomcat    7.0.5                                                                    h1       937.20     1.98    
tomcat    7.0.8                                                                    h1       905.66     1.98    
tomcat    7.0.8+shadowServletServerHttpRequest                                     h1       1531.25    1.97    
tomcat    7.0.8+shadowServletServerHttpResponse                                    h1       1229.97    1.99    
tomcat    7.0.8+shadowServletServerHttpRequest,shadowServletServerHttpResponse     h1       2956.34    1.98    
tomcat    7.0.4                                                                    h2       2987.46    1.98    
tomcat    7.0.5                                                                    h2       952.25     1.98    
tomcat    7.0.8                                                                    h2       923.62     1.98    
tomcat    7.0.8+shadowServletServerHttpRequest                                     h2       1628.64    1.98    
tomcat    7.0.8+shadowServletServerHttpResponse                                    h2       1287.94    1.99    
tomcat    7.0.8+shadowServletServerHttpRequest,shadowServletServerHttpResponse     h2       2912.74    1.99    
jetty     7.0.4                                                                    h1       2157.92    1.98    
jetty     7.0.5                                                                    h1       778.80     1.98    
jetty     7.0.8                                                                    h1       818.60     1.99    
jetty     7.0.8+shadowServletServerHttpRequest                                     h1       1001.91    1.99    
jetty     7.0.8+shadowServletServerHttpResponse                                    h1       1373.84    1.98    
jetty     7.0.8+shadowServletServerHttpRequest,shadowServletServerHttpResponse     h1       2513.35    1.97    
jetty     7.0.4                                                                    h2       2407.70    1.99    
jetty     7.0.5                                                                    h2       781.01     1.98    
jetty     7.0.8                                                                    h2       821.94     1.98    
jetty     7.0.8+shadowServletServerHttpRequest                                     h2       1043.09    1.99    
jetty     7.0.8+shadowServletServerHttpResponse                                    h2       1415.36    1.97    
jetty     7.0.8+shadowServletServerHttpRequest,shadowServletServerHttpResponse     h2       2482.93    1.96