Open
Conversation
in socket assigns to improve performance
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, LiveState encodes both old and new state before diffing. This becomes a bottleneck if the state is updated frequently, e.g. if we're streaming a lot of small updates (LLM response deltas 😄 )
One optimization that can be made is to store encoded old state in the socket assigns so we wouldn't need to re-encode it again. That was initially my plan, but I quickly ran into a wall.
MessageBuilderis a slightly awkward abstraction. Looking at the docs, it's main purpose is to make diffing algorithm swappable, but it does more than that:So I went ahead and merged both of these things into the channel itself and made
JSONDiff.diff/2function replaceable withdiff_functionchannel option. I also made channel not increment version and send empty patch if an updated resulted in an empty diff. There was also a bug inpush_state_changewhich used overridable backend state keys for message that was supposed to be sent to the frontend.Obviously, since
message_builderoption is deleted, this is a backwards incompatible change. So it's completely at your mercy if you want to go with major version.[the code in this PR was slightly LLM assisted]