Howdy!
Last post, I did some preliminary research into how networking can work in games and the different steps that define the network stack of a game.
This week, I will be looking into the networking frameworks employed by two of the most popular game engines: Unity and Unreal Engine.
Unity
Unity's networking framework, known as Netcode for GameObjects is a high-level library that is meant to abstract a lot of the networking logic necessary for making multiplayer games. The goal is to remove a lot of the burden of dealing with protocols and networking frameworks from developers.
Network Architecture
Netcode for GameObjects supports both client-hosted (listen server) and dedicated game servers meaning that there is flexibility for a client to host the game simulation rather than having a dedicated server however, as with all listen server models, this comes with drawbacks discussed last week (host has considerably less latency, the host could cheat more easily, the connection runs through residential internet).
Regardless of whether a Unity game chooses to use a client-hosted server or a dedicated game server, Netcode for GameObjects is server-authoritative with the ability for developers to specify objects that are "client-authoritative" and then validated for legitimacy by the server (so still ultimately server-authoritative)
Transport Protocol
Netcode for GameObjects relies on Unity's custom transport protocol known as Unity Transport (or UTP). UTP provides a connection-based abstraction layer that allows it to be platform-agnostic, making it so that Unity Transport will work for all platforms that can be built to by Unity natively.
Unity Transport can connect to either UDP sockets or WebSockets (for WebGL builds that run within an internet browser application). For UDP sockets, UTP acts mostly like a standard UDP connection by default, but it ensures a connection between client and server. WebSocket connections, by the necessity of the socket protocol, act more similar to what you would expect from a TCP connection with reliable packet delivery and ordering.
Here is a visualization of Unity's network stack using Unity Transport:
As you can see in the visualization, UTP works not only for Netcode for GameObject but also for Netcode for Entities (Unity's networking framework for ECS) as well as any custom netcode that a developer may want to employ.
UTP also allows developers to customize its protocol settings through "Pipelines". Pipelines allow developers to selectively add layers of functionality to the unreliable packets that UTP will send by default. There are a variety of pipeline stages that developers can choose to add to the transport protocol including:
Fragmentation Pipeline Stage: Automatically breaks large messages (meaning larger than the MTU which is usually roughly 1400 bytes) into smaller pieces that are then reassembled on the receiving end
Reliable Pipeline Stage: Guarantees delivery and correct ordering of packets (similar to how a TCP connection would)
Simulator Pipeline Stage: Allows developers to simulate adverse networking conditions locally, such as packet loss, delay, jitter, etc.
Pipelines are a sequence of these stages, specified by the developer. When a message is sent through this pipeline, UTP runs through these stages in order. When the message is received, it goes through the same stages, but in the reverse order:
Here is an example of what a pipeline that employs the fragmentation and reliable pipeline stages might look like:
To be honest, I fell down a bit of a rabbit hole while exploring the UTP and Pipelines and I ended up making a small little example of how pipelines can be used with a custom netcode because I found it interesting. The example below does not use Netcode for GameObjects but rather really simple netcode that I wrote to showcase how pipelines work. I built off of the Unity Transport sample called "Simple Client Server". The original sample was quite simple, it created a connection between a Client and Server through UTP and then sent a packet from the client containing an integer that the server would increment once it received the packet and send it back to the client:
The first thing I did was extend the logic of this sample in order to have the client control a ball on the screen that would move left or right based on player input (left arrow vs right arrow). The client simply sent over the player input and current player position and the server updated the player's position based on the input received and sent that updated position back to the client. Here is what that looked like in editor:
As you can see, the ball moves pretty smoothly but that is because both the client and server are local and are not having to communicate over the internet (thus no latency), so I added a custom pipeline to the SimpleClientServer sample in order to simulate latency with the Simulator Pipeline Stage I mentioned previously:
Above you can see the settings I specified for the Simulator Pipeline Stage including the maximum numbers of packets to send with these specifications, how long to delay each packet for, and what percentage of the packets should be dropped. The pipeline itself is created in the line:
m_Pipeline = m_Driver.CreatePipeline(typeof(SimulatorPipelineStage));The ordering of the stages in the CreatePipeline call is what defines the pipeline flow. Below you can see what the results of this pipeline addition were:
I also added some metrics in order to illustrate what the pipeline settings were doing. My implementations of these metrics are certainly naive but I think it does a good job to illustrate what the pipeline is actually doing:
RTT in this context is how long (in milliseconds) it takes for the client to receive an updated position from the server after a packet (containing input) is sent from the client.
Sent Packets refers to the number of packets that the client has sent to the server in total.
Dropped Packets in this context refers to the number of input packets that were sent from the client but were not received back from the server after some period of time (5 seconds is what I arbitrarily decided). These packets are then assumed to have been lost somewhere in the round trip.
Finally, I added the Reliable Pipeline Stage in order to emulate what TCP-style connection would look like. The only code change I needed to make was changing the CreatePipeline call to include the Reliable Stage:
m_Pipeline = m_Driver.CreatePipeline(typeof(ReliableSequencedPipelineStage), typeof(SimulatorPipelineStage));which resulted in the following simulation:
As expected, with the addition of reliability and packet ordering, our RTT is considerably higher on average but we are not dropping any input packets.
Overall, Unity's Transport Protocol offers a lot of flexibility for developers to customize the transport protocol to their game's specific needs.
Application Protocol
As we had identified last week, the Application Protocol refers to how we identify what information we are going to be sending between clients and the server, specifically with a focus on how to serialize and compress the data we are transmitting. Netcode for GameObjects has built-in serialization for C# and Unity primitive types and also supports the ability to add network serialization for user-defined types by implementing the INetworkSerializable interface.
Netcode for GameObjects also has compression methods available for common data types (such as Quanternion Compression which is used in NetworkTransform objects)
Application Logic
In order to synchronize the data that is shared between clients and servers, Netcode for GameObjects provides three options for synchronizing game states and/or events:
Remote Procedure Calls (RPCs)
Remote Procedure Calls allow one system to call a method remotely on an object that they do not have ownership of. RPCs can be either server RPCs or client RPCs.
Client RPCs are invoked by the server and are executed on all clients:
Inversely, a server RPC allows a client to call a method on an object that is maintained by the server (the server has authority over that object):
NetworkVariables
Network Variables provide a way to synchronize a property between a server and a client without having to use an RPC. When a variable marked as a Network Variable is updated, all clients and the server are updated of the change.
The value of Network Variables is synchronized for both connected clients and newly joining clients.
Custom Messages
Custom messages allow for developers to specify custom messaging behavior that operates outside of Netcode for GameObject's messaging system. These messages are unbound to any GameObject and allow developers the flexibility to define exactly what information they want to be sent through the messaging system.
Unreal
Network Architecture
Similarly to Unity's networking framework, Unreal also allows developers to specify either client-hosted server or dedicated server architectures.
Unreal also follows the server-authoritative model meaning that the server is the authority of the simulation for all clients.
Transport Protocol
Unreal's networking framework uses UDP as its transport protocol meaning that packets are unreliable by default and there is no guarantee provided that packets will arrive in order.
However, Unreal does allow developers to specify reliability on a per RPC basis. This means that the developer has the ability to mark each RPC as reliable or unreliable depending on what is needed. Reliable RPCs are guaranteed to deliver and are held in a queue until they are successfully delivered (similar to a TCP model) while unreliable RPCs do not provide this checking. This allows developers to prioritize certain types of packets over others.
Application Protocol
Like Unity, Unreal provides serialization methods for all primitive types and allows developers to specify custom serialization strategies for user-defined types.
In addition to serialization, Unreal also handles what to send in a way Unity does not with the concept of relevancy.
In Unreal, the server only sends information to clients about Actors that it determines to be in that client's "relevant set". The goal is to limit the amount of info that the server needs to send to each client. These are the rules that Unreal applies (in order) to determine relevancy:
If the Actor is bAlwaysRelevant, is owned by the Pawn or PlayerController, is the Pawn, or the Pawn is the Instigator of some action like noise or damage, it is relevant.
If the Actor is bNetUseOwnerRelevancy and has an Owner, use the owner's relevancy.
If the Actor is bOnlyRelevantToOwner, and does not pass the first check, it is not relevant.
If the Actor is attached to the skeleton of another Actor, then its relevancy is determined by the relevancy of its base.
If the Actor is hidden (bHidden == true) and the root component does not collide then the Actor is not relevant.
If AGameNetworkManager is set to use distance based relevancy, the Actor is relevant if it is closer than the net cull distance
Additionally, Unreal also has a pre-built system for Actor prioritization which allows developers to specify which Actors are more important to replicate first if there is not enough bandwidth to send it all at once (which there never is). Each Actor has a float called NetPriority which corresponds to the amount of bandwidth that the Actor receives relative to all other Actors.
Application Logic
Unreal's application logic is very similar to Unity's. Both have the concept of RPCs which as a reminder are functions that are called locally but are run remotely on another machine. Unreal also has a concept very similar to Unity's Network Variables called Property Replication which allows for clients to be updated whenever a variable changes its value.
In both Unreal and Unity, RPCs should be used for unreliable gameplay events that are fleeting or aesthetic in nature (such as the animation/sound effects taht play when a player opens a chest) and Property Replication (or Network Variables in Unity) should be used for data that needs to be reliable and shared across all clients (such as a player's health).
Conclusion
In conclusion, Unity and Unreal both handle their network frameworks in very similar ways. Both offer either listen or dedicated server network architectures that are server-authoritative. Both operate primarily off of a UDP-style transport protocol with options available for developers to add TCP-like functionality. Unity's solution definitely offers more flexibility out of the box though with its Pipeline system. Both offer ways to serialize and compress primitive types and allow for developers to specify custom serialization methods for user-defined types. However, Unreal also offers more ability out of the box to help developers decide what information needs to be sent (with the concepts of Actor relevancy and priority)
Both handle game state synchronization in similar ways through Remote Procedure Calls and variable synchronization (called Property Replication in Unreal and Network Variables in Unity)
That about wraps up my research into Unity and Unreal's networking frameworks! Next week I will be doing a deep-dive into latency mitigation strategies employed in multiplayer games.
Resources: Unity Multiplayer Documentation:
Unreal Multiplayer Documentation:
Comments