<- The feminine urge to cheat in Splatoon

So, Splatoon 3 just came out recently. I’ve been having a lot of fun, but my favorite part about the game so far is the plaza when booting up the game.

There’s a neat little feature in the Splatoon series where you can draw a 320x120 1-bit image in game, and it’ll pop up in the main plaza for other players around the world to see. There is, however, the time honoured tradition of botting your Wii U/Switch in order to automatically print out images. What’s a user-generated-content without some fun botting?

I set off to hack together my own bot. The end goal? Nothing else but the beloved duck.png!

The final product.

However, I wouldn’t be showing you this now if it was the only goal. Oh no, this is a tale of something much bigger - a tale of painters, processors, and Pro Controllers.

ahem Let’s begin.

It begins with paint

So, like any other bored nerd on a Sunday morning, I started my day typing in “Splatoon image printer” into Google. I quickly found the work of Victrid’s splatplost, a Bluetooth-based printer.

Splatplost worked by using the libnxctrl Python library, which used Bluez (the Bluetooth stack for Linux) to emulate a Pro Controller wirelessly.

My main computer is a Windows machine (libnxctrl uses Bluez, so it’s Linux only), and it doesn’t even have a Bluetooth adapter in it, so I’d have to work with this on my laptop.

After jingling keys in front of my face for an eternity as I waited for everything to install into a virtualenv, I started it for the first time, and…

Thus do I remember the fate of this laptop: a 4K screen with half of the GUI libraries having fucked up DPI. Whew, this is gonna be a bit…

So, I clicked the “Start Pairing” button, and… it did nothing. Why? There’s no logs, so no way to tell!

After bashing my head against GitHub issues for a while, I realized that splatplost’s GUI is relatively new. Before Splatoon 3, it was just a CLI app split into multiple Python scripts.

I decided to clone the repo myself, go back to an older period in Git history (before the GUI), and manually applied the required fixes for Splatoon 3 myself. After doing this, I was greeted with an error in my console:

…drum roll, please…

…hciconfig was required. Here’s a fun fact about hciconfig: It’s so deprecated, not on the official Arch repositories (had to download it through the AUR), and the AUR description literally says “deprecated hciconfig tool from bluez”.

And there was no disclaimer or warning from this. Thankfully, I left a comment on GitHub, and the developer added more logging. Hopefully more Arch-brained users can benefit from this in the future.

With that headache over, Splatplost now worked, but I noticed it was working slowly. See, the thing is, Splatplost has a nice pathing system to try and find the best way to draw the images. However, literally going line-by-line would have been faster for this image, and Splatplost wasn’t optimized enough to know this.

Because I didn’t want to wait here for a projected 2 to 3 hours, I looked at my other options. What now?

Where we’re going, we don’t need pathing

See, the beauty of Splatplost is that the controller part is just a Python library. So we can do it ourselves!

I don’t really write Python often, and when I do, it’s messy quick hacks like these. And I’m perfect at quick hacks like these.

So, I wrote my own painter in about half an hour, using Splatplost as a reference. You can see it here if you’d like, but it’s not very fast (or creative).

I was able to paint two images, but I noticed something wrong with it. Due to the nature of Bluetooth, I couldn’t crank the speed too high or else it would desync and mess up. Even worse, it still seemed to make a mistake every once in a while!

I ended up tweeting about it, and my good friend Aly chimed in:

have you considered just doing it with tas-script or sys-script? I made my Mario Maker 2 painter with nx-TAS (now obselete) but the format is still supported by tas-script

This was an interesting avenue. I have a homebrewed Switch - I actually have two of them, because I bought one specifically for homebrew back before emuMMC existed.

However, that homebrewed Switch doesn’t own Splatoon - and given the amount of, uh, “non-Nintendo-friendly NSPs” that were installed, I didn’t want to connect it to my Nintendo account.

That’s when Aly suggested MonsterDruide’s ArduinoTAS, which begins the start of the hardware rabbithole.

Creating tool assisted paintings

ArduinoTAS is a fork of wchill’s SwitchInputEmulator, with a bit of Python script glue to support running a TAS script. The idea is to simulate a Pro Controller over USB using an Arduino.

If you’re not familiar with what a “TAS” is - it means “tool assisted speedrun”. This is specifically designed to play back a list of inputs in a specific order and timing, designed for speedruns performed by a computer to display the theoretical perfect speedrun.

The idea is simple - set up ArduinoTAS, write a program that takes an image and produces a TAS script, and then play that script on the Arduino. The real challenge is setting up ArduinoTAS.

ArduinoTAS isn’t actively maintained, so much so that the images in the README were dead. With little knowledge of how an Arduino really worked, I gathered one from the depths of my technology bin, and set out to understand the README.

ArduinoTAS had two interesting components to it: it required a USB to UART adapter (to send scripts over - the USB port for the Arduino was being use to emulate a Pro Controller), and it required a HDMI to VGA adapter. You would wire up the VSync pin from the VGA adapter to the Arduino, so it has an accurate reading of when each frame starts. That’s some really clever black magic.

After a long while of troubleshooting, I eventually figured out what cables to connect and where. I ended up using a separate Arduino instead of a USB to UART adapter, because I had one lying around and I didn’t want to wait 2 days for the package to arrive.

I didn’t have a soldering iron upstairs, so I just kind of touched the wire to the right pin using the power of friendship:

Check this cable out!

Now, all I needed to do was write a program to print out the images. And then I had an idea.

“What if you could emulate a controller and use a keyboard and mouse to play Splatoon?”

Cheating with the power of Python

In a haste of motivation, I quickly copied the clientTAS.py from the ArduinoTAS repository, and installed pygame into my virtual environment. It was time to make something sinister. By the way, at this point, I had finally set up my dual boot and was writing this code on my desktop.

Using the power of 10 if/else statements, I hacked together a proof of concept:

Check that out! Nifty, huh?

Of course, I’d instantly get banned if I queued into a public lobby with this. Not that I could, anyways - inputs were being dropped and the camera wasn’t precise.

ArduinoTAS, though very useful for painting, wasn’t very useful for cheating in Splatoon. Instead, I’d head upstream for that!

Swimming up the stream

As said before, ArduinoTAS is just a fork of SwitchInputEmulator, which is probably more suited for the task of emulating controller input. Let’s take a crack!

After a short flash later, it seems to work, but we need to get a script going. There exists a InputServer we can use in the repository.

It’s a C# server, targeting .NET 4.7.1. Updating the dependencies, changing the target framework, and changing the serial path is enough to get it building on Linux.

There seems to exist some sort of WebSocket functionality to control it, but it seems geared around Twitch, which I’m definitely not doing.

There also exists a “MultiInput”, which is a QT app. I kid you not, they don’t ship a Makefile or anything useful - they ship a Linux binary precompiled in the source tree. Jesus Christ.

I mean, it runs, but all the inputs are messed up and I have no idea how to build it. Given that’s both ways of interacting crossed out, I seeked to write my own client in Rust, but it ended up just stalling and I couldn’t figure out why.

Thus, did I stumble across the final part of this journey:

The Bluetooth ouroboros

You know, I originally moved to the Arduino, because Bluetooth was unstable. And now that my goal isn’t precise printing, and just controlling the game with a keyboard & mouse, Bluetooth looked like a good option!

One trip to Micro Center later, my PC now has Bluetooth support. I stumbled upon nxbt, another Python library using Bluez to emulate a Pro Controller - this time, with an already working web app!

Installing this was a nightmare. The dependencies were messed up and broken on Python 3.10, so I had to downgrade some things manually. If I wasn’t using a virtual environment I might have cried.

The web app, sadly, did not have any rebinding features. That’s okay - it’s a library! Following in the footsteps of ArduinoTAS, I hacked together another pygame app. And it literally looked the exact same, so much so that I won’t even bother to include the (80 megabyte) video of it!

The mouse was more choppier, strangely, but the inputs felt a bit more responsive (which is funny, for having more latency). Maybe I can fix this with a rewrite in not-pygame.

I also considered trying to emulate a controller with keyboard/mouse inputs, and just use the web app, but I couldn’t find anything that did that on Linux.

Conclusion

That sure was an ADHD spiral if I’ve ever seen one.

If you’re looking to print out images - use Splatplost or Shiny Quagsire’s Arduino printer - I ended up using the latter after I got bored of making my own, and it seems to work somewhat well. If you’re looking to cheat in Splatoon - please don’t. This was purely for my curiosity and I don’t plan to use this in online matches. Pls no ban Nintendo.

Shoutouts to all the developers making all the code referenced in this post - you’re powering my skid dreams. Thank you!