June 5, 2017
DIY MPU 9265 Arduino-Unity Motion Controller
A do-it-yourself electronics program that communicates accelerometer data to a game built in Unity!
Built for: GAM-51: Game simulations and mechanics.
Project Development Time: 3 weeks
Software/Engine: Unity3D, Arduino IDE
I've always been interested in both video game development and microcontroller-based electronic projects. This project combines my two favorite hobbies and was an awesome learning experience along the way. This article will serve as a guide through my development process and will allow me to share with you some of the challenges I faced while creating this project.
My controller design features haptic feedback vibrations, RGB lighting that matches the in-game background, and an accelerometer for motion controls. There's also two buttons, one for firing your grappling hook, and calibrating the sensors, as well as a potentiometer for zooming in and out.
Part I: Getting Unity to talk to an Arduino over a serial connection
A couple of years ago, I attempted to write my own C# <-> Arduino serial communication system so that I could interface my electronics projects through the Unity game engine. It honestly didn't take too long before I had a basic proof-of-concept program that could set pins high and low on an Arduino board, but my original system was not very robust, reliable, or even intuitive to use. It was a neat experiment, but it was ultimately shelved.
In my final semester in the game program at my college, I felt that I needed to end my four year journey with something special - something that hasn't been done before at this school. My professor at the time previously expressed a great interest in hobby electronics and Arduinos, so I thought, "This is a perfect chance to create something that I love to work with too!" I forked my previous attempt at a C# serial communication system and began work on the "new and improved" version for my final project immediately.
i. Writing to Serial Ports with C#
C# actually has pretty great support for serial ports right out of the box. The problem is, Unity, which uses the Mono compiler, does not fully support all of the great .NET serial port features yet. Namely, the event that is fired whenever there is incoming serial data does not work. Because of this, I was forced to manually connect to the Arduino, verify that the device I'm talking to is the Arduino controller, and poll for incoming serial data myself.
Let's begin by brute-force connecting to the Arduino and sending our first command over serial. All code examples assume you will be working in Unity/C# and the Arduino IDE.
Create a new C# script (I called mine "Interface") in Unity and attach it to any game object.
I should point out that there is a downside to sending commands to the Arduino. When you send a command, it will interrupt the incoming data stream until the transmission is complete, blocking out any further input from the Arduino. You only want to send commands to the Arduino during lower-priority times like handshaking and initialization.
ii. Reading Serial Data on the Arduino:
Begin by creating a simple LED circuit. If you would like to use the built-in Pin 13 LED, feel free to do so. In this case, I have connected an LED to digital pin 2 and ground with a 220Ω series resistor.
What we're going to do on the Arduino end is to listen to the serial port and test to see if we've received the "Hello World" message from Unity/C#. Once we receive the correct message, we'll light up the LED for five seconds, then turn it back off again. Enter the code down below into the Arduino IDE, upload it, then play your game in Unity. If everything went to plan, you LED should light up! Make sure to watch your capitalization on "Hello World" and be sure to check your port numbers (Same port you use to upload your code).
Part II: Getting an Arduino to send data to Unity
i. Creating a serial polling system in C#
As I mentioned before, there is an event that you can subscribe to in C# that fires whenever incoming serial data arrives; however, this feature does not function properly with Mono, so we must create our own serial polling function. Jump back to your Interface script in Unity and replace it with this:
One thing to note is that we no longer automatically close the serial port. If you can not program your Arduino because access to the serial port is denied, simply run your Unity game again and hit the "Escape" key to close the port, then stop your game.
ii. Sending serial data from the Arduino
This part is relatively easy compared to the C# code above. If you've ever worked with Arduinos before, you'll already be familiar with the Serial.print(); family of functions. All we're going to do for now is to continuously send serial messages to Unity along with a count of how many times we've looped.
Connect your Arduino, fire up your Unity game, and check the Unity console. You should see some verbose information on the incoming serial data.
In the interest of brevity, I won't explain the entire serial communication system that I created for this project. Just understand that the "handshake" system that I created is built on the code that you see above. Unity sends over a serial command to the Arduino that says "DISCOVER" and if the Arduino is on the other end, it will reply with "ACKNOWLEDGE." Once this transaction occurs, the Unity Interface will change to a "Connected" state and will begin transmitting and receiving serial commands. You can browse my final Interface script here. and the final Arduino sketch here.
Part III: Accelerometer data from the MPU9265 Module
The MPU9265 is a three-axis accelerometer, gyroscope, and compass module that we can read using the I2C interface. If you're unfamiliar with I2C, don't worry, most of the hard stuff is taken care of by the Wire.h library included with the Arduino IDE. You can also find similar accelerometer units under the names MPU 6500, MPU 6050 and MPU 9250. I'm not sure if they would be completely compatible with my software, but they're all similar enough that it shouldn't be too much trouble to port over code should there be any differences.
Keep in mind, these MPU sensors run on 3.3 volts, not 5. On my module, there are current limiting resistors on the I2C lines and a built-in 3.3v regulator on VCC. So while technically 5v would be fine on VCC, I recommend connecting VCC to the 3.3 volt pin instead to avoid any accidental magic smoke.
Since we are using I2C, we need to connect our sensor to two specific pins on the Arduino. If you're using an Uno, these pins are marked SDA (Serial Data) and SCL (Serial Clock). These lines are also directly connected to pin A4 (SDA) and A5 (SCL) if you'd prefer to use those pins or are using a Nano like me.
Go ahead and follow the wiring diagram to the right and connect the MPU SDA pin to Arduino A4(SDA) and MPU SCL to Arduino A5(SCL).
Before we begin programming this sensor, I'd like to link a project that gives a ton of great example code for this sensor.
Add the following code to your Arduino sketch. This will only give us the accelerometer data from the sensor.
Once you've added the above code, check out the Arduino Serial Plotter (Arduino IDE-> Tools -> Serial Plotter) to see a graph of the X, Y, and Z axis values from the accelerometer. Give your prototype board a shake and a turn and watch what happens to the graph.
Part IV: Sending accelerometer data to Unity
Now that we've successfully read the MPU module, it's time to do something useful with that data. Since we're dealing with rotations here, we need to begin by formatting our raw sensor data into something a little more Unity-friendly.
Unity's internal rotation system is Quaternion based, but the numbers exposed to the developer use the friendlier Euler rotation system (0-360 degrees).
Our MPU sensor gives back values that really aren't compatible with either of these systems. Values from the MPU sensor usually range between -3000 to 3000 on each axis. We can use a handy built-in Arduino function to constrain these values to 0-360 for us though. The "map()" function.
We'll also create our own Arduino Vector3 object using structs to help keep our Data organized. Just a word of warning though, this is where things start to get a little trickier.
Let's head back to the Arduino IDE for a moment and update our code to be a little more Unity-friendly, then send out some accelerometer data in a package that can be read by our Unity Interface script!
Upload the above sketch and jump back to Unity. We'll demonstrate our new accelerometer data by rotating a cube.
Create a new cube in your Unity scene and place it in front of your camera. Now that we've sent over formatted accelerometer commands from the Arduino, we need a way to parse them. C# has an excellent way to split up strings based on delimiters. If you look back up to the Arduino code, you'll notice that we insert a ":" before we send the next rotation axis value. These ":" are our delimiters and we can use them to split apart strings into individual values. Fortunately for us, C# has great built-in support for splitting strings based on delimiters. You'll all of the string magic happen within the "ParseAccelerometerData()" function. Open up your Interface script in Unity and add replace it with the following:
If you look the on object where you've placed this interface script, you'll see an open "Transform" slot. Go ahead and drag your cube into this slot. Then, fire up your Arduino and press play in Unity. You should be able to rotate the cube around using the controller that you've built!
If you play around with your controller a little more, you'll notice that it's matching your movements... but... something doesn't feel right. It's almost as if some of the axis are off or the cube is rotating in an incorrect way. This may be due to a phenomenon called "Gimbal Locking" which is one of the main disadvantages of using Euler angles.
Selecting a single axis to work with seems to work fine for most 2D games, but 3D rotation might be hampered by incorrect rotations. If you're willing to get clever with your vector math, you can probably find a way around it.
Experiment with it a little and see what you come up with! Add a couple more features to your controller to make it special!
Part V: Ending Notes
So that was my high level walk-through on how you can interface an Arduino and an accelerometer sensor to the Unity game engine. You'll notice that my finished controller also has buttons, RGB lighting, a rumble motor, and a zoom-control potentiometer. Once you've mastered the above code, all of these extra additions are trivial because they all rely on the same methods of transmission. If you would like to build the complete controller and interface yourself, you can follow the full schematic here and clone the full source code and Arduino sketch for the game that I made that utilizes this controller.