<- How I made Bomb Rush Cyberfunk multiplayer

On August 18th, Bomb Rush Cyberfunk released on Steam. On August 23rd, I made my first commit to Slop Crew, a multiplayer mod I made for it. Let’s talk about how I made it!

Keep in mind that the technical details here will be talking about the Slop Crew 2.0 rewrite, which has vastly different netcode compared to the original 1.0 (which was a WebSocket transporting a custom binary format, which sucked).

1000 degree glowing knife versus UnityPlayer.dll

BRC is a Unity game. In Unity, the code you write is a set of C# scripts and classes, which get compiled from their source code when the game is built. This compilation can output in one of two ways, depending on what method is chosen by the game developer.

• Mono: The C# code is compiled into the Common Intermediate Language (which I’ll just call IL from now on), and is evaluated while the game is running by the Mono runtime (the C# runtime Unity uses).
• IL2CPP: The C# code is compiled into IL, but that IL is then transformed into C++ code, which is compiled into a native executable instead of being interpreted on the fly by the Mono runtime.

IL2CPP is significantly harder to reverse, given it becomes a native binary; you’d have to use a decompiler like IDA or Ghidra to understand how the code functions. Thankfully, Team Reptile chose to use the Mono runtime for BRC, meaning it is significantly easier to reverse. Tools like JetBrains dotPeek can convert the IL code back into C# - an approximation, albeit - which makes understanding and interacting with it significantly easier.

This allows us to read what the game is doing, but what about modifying it? We can use a plugin framework like BepInEx, which allows us to write C# code that has access to the Unity engine/game specific code, and then load that code at runtime when the game starts. We can use a library like Harmony to modify the game’s code, as well - without even having to touch the game files!

Wrangling hundreds of players efficiently

Inside the game’s code, there exists a method called SetupAIPlayerAt. This method creates a player object normally intended for the game’s AI, but we can instead hijack it to create a player for us, and then puppet it ourselves. But first, how are we going to communicate with other players?

Slop Crew implements a client-server model, such that multiple clients (in this case, multiple instances of the Slop Crew BepInEx plugin) connect to a single server, and the server exchanges messages between them. The client and server are both written in C#.

For networking, Slop Crew uses Valve’s GameNetworkingSockets, a library created as a standalone alternative to Steam’s networking systems that can be used in developers’ games. It is more intended for people making new games rather than trying to staple multiplayer onto a singleplayer one, but it’ll work. GameNetworkingSockets (which I’ll just call GNS from now on) is written in C++, but our plugin & server are written in C#. I found a C# wrapper around it on GitHub and modified it to my needs.

This has a problem in that the game is Windows-only, so the plugin will only ever run on native Windows or through Wine; but the server needs to run on both Windows and Linux natively. Thankfully, with a little bit of hackery to specify what platform it’s being compiled for (related to struct packing), we can make it run fine on both Windows and Linux.

We have a way to transport data between the plugin and server, but what data will we transport? In this case, we need a protocol, which I settled on Protocol Buffers (or just Protobuf for short). I can write my network schema in Protobuf, and then generate C# code to use that schema with. This means I could’ve picked other languages for the server, but I felt that writing it in C# was the best option (also because the 1.0 server was in C# as well).

Everything is a state machine if you look at it hard enough

We have a transport and protocol, but we need to exchange data between the plugin and server, along with managing their state on both sides. For this, I made three files for each part of the network: shared data, messages sent to the client, and messages sent to the server. This allows me to define a player in the network like this:

message Player {
  optional uint32 id = 1;
  string name = 2;
  Transform transform = 3;
  CharacterInfo character_info = 4;
  repeated CustomCharacterInfo custom_character_info = 5;
  bool is_community_contributor = 6;
  optional string representing_crew = 7;
}

message ServerboundHello {
  Player player = 1;
  optional string key = 2;
  int32 stage = 3;
}

When a player connects to Slop Crew and loads into the game, it sends a ServerboundHello packet to the server, which contains information about the player and the stage they’re in. The server groups players together by stage, which is just a number that corresponds to the ID inside the game. After assigning the player to a stage, it gives an updated list of all players in that stage to all players in that stage (effectively a list of players sent to every player in that list).

When the plugin receives this packet, it looks through to see what players are new/missing/updated, and handles it appropriately. The plugin creates an AssociatedPlayer class, which effectively ties the player object in the scene to the player object in the network.

Whenever a player’s character does something like play an animation or switch outfits/characters, it sends a packet to the server that then relays that packet to everyone else in the stage. For example, the PlayAnim method in the game’s code is hooked with Harmony to detect when it’s called for the current player, and then sends a packet to the server to relay to everyone else, playing the animation on everyone else’s client.

Managing score battles and races

The game has a concept of an “encounter” internally, for stuff like score battles in the game’s story. Slop Crew has its own encounter system, where both the client and server keeps track of its state. We use the in game phone to start encounters (which we were the first mod to add a custom phone app to!), which we hijack by simply hooking when it’s initialized and adding a custom app (along with bashing an app entry into the home screen).

When a player requests to battle another player, the server keeps track of that request and sends a notification to the other player. If the other player responds to that notification (sending another request), the server will detect both players have requested to battle each other, and put them into an encounter. Encounters have state managed on both the client and server that gets synchronized, but usually things like timers run independent of each other. When in a score battle, the plugin sends a packet with its updated score to the server, and the server routes that packet to the appropriate running encounter and handles it (updating the score and telling the other player).

Races are slightly more complicated in that the server also picks a random configuration for the race and serializes it over the network, which the plugin then spawns checkpoint objects into the game scene with.

Getting data in and out of other plugins

Slop Crew has a separate API assembly which other BepInEx plugin developers can use to receive info about the current connection. This also exposes an API to let people send custom packets to the stage, to add integration into their own plugin.

Slop Crew also uses some other plugins’ APIs, too - notably the CrewBoom API, which is used for syncing custom characters. Note that custom characters are only applied if you have the same character installed locally (would rather not repeat some FFXIV moon related mistakes).

Running more services on it, too

The Slop Crew server also runs an HTTP API with ASP.NET Core inside of it. This is used for linking your Discord account to your in game character - the server implements the Discord OAuth flow and then generates a unique key to insert into your config file. The frontend for the website is written in React and talks to the API running on the server, which saves information to a SQLite database. When a player connects and their plugin provides an authentication key, the server looks it up to find the account tied to it.

The Slop Crew server also has support for Graphite metrics, which I use to connect to a Grafana instance for player activity. This component is completely optional, but it’s fun to stare at the statistics. Right now, it only tracks population of a given state, total connection count, and how many people are using a certain version of the Slop Crew plugin.

Ensuring everything is safe & hostable

The Slop Crew server is built on GitHub Actions and uploads a Docker container to GitHub Packages. The plugin is also built on GitHub Actions, and automatically does releases to GitHub and Thunderstore when a new tag is created. This means that the Slop Crew plugin needs to have access to the game files in order to build against the game code. To solve this, I use the BepInEx assembly publicizer to create a stripped down version of the assembly, with no code in the methods - effectively equivalent to a header file. This is downloaded from a web server when the plugin is built.

Slop Crew is 100% open source software, and I try my best to make it so that anyone can selfhost it. The Docker container is what’s used in production, but anyone can use the container and modify the Docker Compose file.

Conclusion

Hopefully some of you enjoyed that. Slop Crew is a pretty big monolith of code now - over 10,000 lines, apparently! - but it certainly doesn’t feel that way to me. At its core, it’s a very simple set of systems connected to one another. If you liked reading that, but haven’t played the mod before, consider hopping online sometime!