Progressive HTML rendering support [SPR-14981]

[spring-projects-issues]

Sébastien Deleuze opened SPR-14981 and commented

We should provide a way to change flushing strategy without using ServerHttpResponse#writeAndFlushWith(Publisher<Publisher<DataBuffer>>) low level method.

That could be via supporting Publisher<Publisher<?>> return values or introducing a new annotation that could allow to flush the data after each element (could make sense when you serialize POJOs).

---

Issue Links:

• Provide a way to enable streaming mode via annotations (and eventually app config) [SPR-15103] #19670 Provide a way to enable streaming mode via annotations (and eventually app config)

3 votes, 8 watchers

[spring-projects-issues]

Sébastien Deleuze commented

Not sure we should do something here since after #19671 and this commit media type is used to enable flushing by element or not + performances have been improved. Annotation based programming model is quite high level and it is possible to use ServerWebExchange to implement custom low level flushing behavior, so I think it is probably better to wait more feedbacks before eventually doing something.

[spring-projects-issues]

Sébastien Deleuze commented

Notice that I have added an additional commit that enable flushing by element for application/stream+json mime type.

[spring-projects-issues]

Florian Stefan commented

I think it would be really useful to be able to control the flushing behaviour per controller method. A concrete example for this would be "Progressive HTML rendering". The idea is to split up the HTML into chunks and send each chunk to the browser as soon as it is available. So the browser can start rendering the page and also download additional resources without the overhead that AJAX via HTTP /1.1 introduces.

Facebook is calling this "BigPipe". We implemented this using the Play framework (see here for the basic ideas).

It would be really cool to do something similar with Spring 5 and Reactor. Having control over flushing behaviour would definitely help.

[spring-projects-issues]

Dave Syer commented

That's a really useful link (the Play talk). I think it highlights the point that actually, it's not really the controller that needs to know about flushing, but the view renderer. Thymeleaf already has a weak form of this (it's still a bit leaky, and you can only drive the flushing behaviour using a single Publisher). Maybe this is getting off topic from the original issue though (which was more about how to flush responses based on the body type).

[spring-projects-issues]

Sébastien Deleuze commented

We are getting a little bit out of the original topic indeed, but I think that's a good thing to understand the context and various alternatives to correctly evaluate what to do here.

I do think progressive HTML rendering would be a very useful feature. By progressive HTML rendering I mean being able to render sequentially templates based on multiple asynchronous model attributes without waiting the resolution of all these attributes, this is super useful for mobile and keep good SEO while avoiding users to wait few seconds to see anything. The most common use case would likely be request remote JSON HTTP endpoints with WebClient and begin to render and flush the HTML to the client asap the first async model attributes complete. WebFlux has the underlying architecture to support it, but there is some missing pieces when you consider the feature end to end. This is obviously not magic and model attributes usually coming with the more latency should be placed at the end of the template.

As pointed out by Dave Syer, in Spring world this should mainly be solved at view rendering level. Reactive template engine supporting progressive HTML rendering should override AbstractView#resolveAsyncAttributes to avoid pre-resolution of async attributes and pass them as they are to the engine to allow it listening async attributes completion. Since a single stream of HTML is produced, such template engine should determine what is the order or resolution based on the templates. There is currently not yet support for that even in Thymeleaf, so this is open for contribution and experimentation.

That said, it is also possible to implement that in a low level fashion by creating a Reactive pipeline that will call manually various codecs and output a Flux<DataBuffer> and call ServerHttpResponse#writeAndFlushWith(Publisher<Publisher<DataBuffer>>). This would be quite complicated, and I understand here the need to support for example flushing per element when returning Flux<String> (for example when you request remotely rendered webpages.

Arjen Poutsma Rossen Stoyanchev Brian Clozel Do you have any proposal about how we could specify this flushing on next element? Maybe a dedicated annotation or annotation attribute for the annotation programming model and dedicated methods or codec hint for the functional API ?

[spring-projects-issues]

Dave Syer commented

A Rendering kind of has some of the pieces that you need to encapsulate a fragment that needs to be flushed (e.g. comparing with the Play example where sections of the screen were built up and flushed as they became available). Maybe what I need is to be able to populate a Model with attributes of type Publisher<Rendering> (or something), and have them somehow flush the response when they are delivered?

[spring-projects-issues]

Sébastien Deleuze commented

Dave Syer If we are talking about the same use case, ie. being able to assemble fragment of HTML remotely rendered (for example by a remote Markdown rendering webservice or a remote template rendering server), it seems to me Mono<String> model attributes would be enough at least for a start (we can imagine having more feature to escape HTML, etc.). What is missing is "just" a Reactive template technology with a View implementation that overrides AbstractView#resolveAsyncAttributes and build itself the reactive pipeline in the right order based on templates and flush that data after each onComplete event (or onNext if we go as far as supporting rendering multiple Flux<T> like Thymeleaf DATA-DRIVEN mode). Thymeleaf 3.1 could be a good candidate, I will talk to Daniel Fernández about that.

[spring-projects-issues]

Sébastien Deleuze commented

I want also to clarify there are 2 use cases:

• One purely server-side which is SEO friendly but where the aync elements will rendered in order. I think with proper advanced CSS styles and HTML structure, you could also achieve predictive displaying of your data regardless the order in the DOM, so the Reactive templating could under some conditions implement a "send first completed" behavior.
• One implying JS code and Ajax or SSE to inject the HTML fragment as they are available. A single SSE stream opened to push fragment seems to me the best way to implement that, but regular Ajax request would be OK as well since we now have HTTP2 which supports many concurrent parallel requests.

I think both are interesting and cover different use cases.

[spring-projects-issues]

Dave Syer commented

I think there is a third possibility which is render a single stream and use JS to position the elements as they arrive (you don't need Ajax or SSE, so there's less code in the browser potentially). I'm interested in what the CSS options are though - maybe I'm already doing it, but with no custom CSS (browser seems more than willing to show partially complete HTML, e.g. table rows as they flush, without necessarily needing to see the end of the table).

FWIW I also don't think Publisher<String> is a very nice model (if you mean that the string is HTML), although it might be a stepping stone on the way. I want to be able to compose server side views with templates.

[spring-projects-issues]

Sébastien Deleuze commented

Dave Syer I don't understand the third way with single stream + JS, could you please give more details about it?

What would be the limitations of Publisher<String> model attributes regarding to server side views with template composition? What do you expect instead?

[spring-projects-issues]

Dave Syer commented

could you please give more details about it?

E.g.

<script id="foo" type="text/html">
// render content here
</script>
<script>
  $("#content").append($("#foo").html())
</script>

What would be the limitations of Publisher<String> model attributes

You would have to render all the strings yourself before adding them to the model, whereas Spring has a nice abstraction for doing that via a Rendering.

[spring-projects-issues]

Sébastien Deleuze commented

Thanks for the example of the third way (even if I am not sure that will work).

You would have to render all the strings yourself before adding them to the model, whereas Spring has a nice abstraction for doing that via a Rendering.

I still don't get it, to me there is 2 main use cases:

• You are getting data asynchronously with something like WebClient receiving DTOs via JSON and you put your Flux<Foo> or Mono<Bar> in the model via Rendering for example. Data come remotely with latency but template rendering is done locally
• You are getting HTML fragment already rendered from a remote Mardown webservice when you send POST request or a remote Thymeleaf or whatever other templating technology, you get various Mono<String> attributes that you can compose with your locale template engine via using Rendering as well.

In both cases the current Rendering is use to pass Mono and Flux model attributes. The missing piece is only the template rendering implementation that will not wait all async attributes to be completed to begin rendering.

Is your use case different?

[spring-projects-issues]

Dave Syer commented

Is your use case different?

I think so, but maybe I'm asking for too much. I'm trying to generalize the Thymeleaf features, and copy the guy using Play from LinkedIn to some extent, so the fact that those two examples exist gives me some confidence that we can do better.

If the remote call returns a Flux<Foo> and it can be rendered as a fragment, incrementally, then the HTTP response can be flushed as Foos arrive, not after they all are ready. Similarly the page might contain content from a Mono<Bar> and a Mono<Spam> and I want to render them as they arrive, as fragments (with a View and everything).

As I already said, I think this will require changes in the view template renderer. But hopefully Spring can provide a framework for doing that in a sensible way.

Remote rendering with a markup service is still a nice option. It just isn't the same, and gets a bit awkward if the rendering is actually not remote. Or maybe not. I'm happy to be proved wrong.

[spring-projects-issues]

Sébastien Deleuze commented

Yeah with Flux<Foo>, Foos could be rendered as they arrive.

Maybe at some point we will be able to provide some higher level constructs to help with that, but not sure since this is tricky and very related to each view rendering technology.

In any case, I still think the first steps are implementing these ideas in one of the template rendering running on top of WebFlux like the upcoming Thymeleaf 3.1, and on our side provide a way to flush data after each onNext in order to allow developers doing powerful things programmatically.

[spring-projects-issues]

Dave Syer commented

So this works quite well already:

@GetMapping("/flux")
@ResponseBody
Mono<Void> flux(ServerWebExchange exchange)
          throws Exception {
     return exchange.getResponse().writeAndFlushWith(Flux
               .just("<html>\n<body>\n", "<h2>Demo</h2>\n", "<span>Hello</span>\n",
                                 "</body></html>\n")
               .delayElements(Duration.ofSeconds(1))
               .map(body -> buffer(exchange, body)));
}
private Publisher<DataBuffer> buffer(ServerWebExchange exchange, String body) {
     return Mono.just(exchange.getResponse().bufferFactory().allocateBuffer()
               .write(body.getBytes()));
}

Kind of rough, but it shows the principle, and the browser renders the page progressively. Question: it doesn't work if you don't end the fragments with \n (only complete lines are flushed). Is that expected? Why?