Networking
The design of cafe is inspired by the model-view-controller (MVC) pattern. The diagram below is a high level overview of the game network architecture. The idea is to centralize command and control of the game state through Controller objects but allow the Viewers to passively be controlled by the Model. This pattern fits the use cases usually imposed on developers by convenience tools. In other words, the hope is that this pattern provides the best of all worlds.
The Model itself is duplicated on the client side and the server side with various synchronization mechanisms planned to support any type of networked game play required. For completeness, the design mandates that the service is authoritative at all times with the acknowledgement that the client will be operating ahead of the service at all times due to latency. If cafe requires a real-time handling of client actions in the future, the plan is to simply store a history of state and allow the client to take action based upon the most recent latency metrics gathered. That said, none of this is required at this time and will be ignored in implementation for now.
In summary, we've developed a nice symmetrical pattern that looks like:
Viewer <- Model <-> Controller <-> MsgQ <-> Controller <-> Model -> Viewer
Below is a brief break down each of the components along with their responsibilities, roles, and communication paths.
The message queue is where all communications occur between all services and clients. Although the message queue will always be "closer" to the server, by design it remains an independent entity that both servers and clients connect to. In other words, all server instances and client instances will "connect" to the message queue's "listening" port for all communications.
Another optional API that all cafe clients must observe from the message queue is the API that controls the transition from one message queue to another. This requirement ensures that the system can scale in the future. The way this works is that users will connect to a gateway message queue and then a server may instruct that client to reconnect to another message queue service somewhere else.
Since both cafe clients and cafe servers connect to the message queue as network clients of the message queue, it is paramount that cafe client users do not connect to a message queue as a server. Roles have been defined for the message queue to prevent any spoofing as a service. All services will authenticate as a service with the message queue to ensure that game clients may trust the information they read as coming from a service. In this model, the message queue itself is responsible for enforcing these roles and any required authentication.
At this time, there are 3 defines communication paths that the message queue supports:
- Broadcast Channels - Public channels where the client may request public information about the room or more commonly where the service will broadcast state updates that all members should be aware of. This channel is public and all communications made on this channel are seen by all members of a "room".
- Unicast Channel - A private channel that is automatically established with the service whenever a client joins a Broadcast Channel. This is where the client may have private communications with the registered service. Another secondary role of this channel is to allow the request of a private channel registration.
- Private Channel - A client controlled private channel to allow a client to client or client to party communication that doesn't need to involve the service. The general idea is for a client to request the channel and then only members within the channel can invite other members.
The original message queue was tightly coupled to the server. It makes more natural sense, but doesn't scale as well. When re-imagining the message queue implementation, there was confusion about how subscriptions work and how to deal with the different roles of service vs client and what is the best way to reference this different types of subscriptions. Previously, everything was stored in a simple lookup table based on the client id. Trouble is that with the current architecture, there is the complexity of the fact that a client id may be the same for the message queue, service, and client if they are all in the same process.
The solution is to just use more channels. Channels already provide a simple multiplexing capability to the message queue and we should just use more of them. The original message queue managed channels by String. A better way to manage these is with opaque pointers and then simply alias the broadcast channels for easy lookup. Aliasing the broadcast channels also provides a great hooking point for matchmaking or redirecting without explicitly having the client connect to a new service (for smaller installations). Each client and service can manage the channel opaque pointer they are a member of and the message queue will enforce members only to prevent random chatter and lurking from outside members.
The primary purpose of the server controller is to facilitate the serialization of responses and parsing of client requests. In general, all client requests are parsed and sent to the Server Model for handling. The server will also broadcast critical state information to all relevant users based on signals thrown or a periodic updater.
In addition to the processing of requests, responses, and broadcasts, the server controller is also responsible for arbitrating administrator controls. Administration controls allow an administrator to query or inject arbitrary updates to the Server Model.
The Server Model holds the critical state of all objects in the scene graph as well as the recent history of all relevant state changes. The server model also serves as the authoritative physics engine of the game state. Generally, the design is that when a client action requests comes in, the server performs the request and eventually responds with the update. While the server model was processing the request, the client has gone on to make other requests asynchronously. Several considerations are made to synchronize all of the interactions between the clients:
- When clients perform actions on other clients, the action is performed based on the state of the scene graph at the time of the incoming action minus the client's latency.
- When clients stop moving, within a given threshold, their position is updated to the server position. Note: This assumes that the client collisions are a close match to the server collisions.
All viewers in Godot are the result of the state of viewports and camera settings from the scene graph. Although more advanced setups can be designed in at a later date, it is assumed that there is only one viewport per cafe process.
One of the key concepts with cafe is the ability for friends to be able to play multiplayer where ever they are. This means that cafe needs to employ a means to allow users to discover themselves. The way this is done can change from environment to environment.
The only good way to control internet gaming is to have a centralized service (like an AWS service) that controls all of the traffic and game state. Discovery will be made by using a well known ReST endpoint.
This is the environment that will initially be targeted. The basic premise is to start a server and beacon an advertisement out to any connected networks on a known UDP port. Each beacon will be signed with a sort of HMAC-SHA256. The user can optionally set a white list to only accept authenticated client messages.
Without a local network to connect to, some users may be required to setup their own ad-hoc networks. Once an adhoc network has been setup, all of the same behaviors found in "Local Service Discovery" should apply. A future version of this should assist the user with setting up the tethering as much as policy will allow.
string - [length (u32)][data]
entry_count (u32) - Number of service entries
entry-N - [string] - URI that indicates a service to connect to
* udp://<ip>:<port>
* tcp://<ip>:<port>
* https://<ip>[:<port>]
payload - <entry_count>[entry-N]
packet - <payload><HMAC(key, payload)>
TODO: Design a solid RPC protocol.
TODO: Document the Godot RPC mechanism here.