[11 Interface and Application Programming]

Final weekly project! Final tool for the toolbox - how do we actually display and interact with all this cool stuff we've learned? This week we needed to write an application that interfac3ed with a user with one of the input or output devices that we made. And then to compare as many of the tools we've been shown to do this as possible. I wanted to learn the graphics engine - Unity - to do this. Unity is the engine I'm using in my thesis research for the VR mission control centre being designed for lunar rover control, so of course getting a headstart here on learning how to use Unity will be very useful for me. So, while there are some other great graphics engines out there (like threejs), I went with Unity.

Working in Unity

Although I have some programming experience, I don't have a ton, so using something like Three.js, is a bit like going to Spain with only basic Spanish and trying to run a high-level business meeting. Pretty challenging right off the bat. Unity is great because you're doing similar things but working through a gui that makes it a bit more intuitive and interactive. Lots of drag and drop. Still not particularly easy though. Going through many a YouTube video and online tutorial, I found this one that was extremely useful. I started by just trying to get one box to represent a finger to rotate around a 'joint' when the signal from my glove told it to. After some playing around with delay time between reading the analog input, anchoring the 'joint' I got it to work! Here are some lessons learned:

By having the 'finger' be the child of the 'joint' the child will rotate around the joint, rather that rotating around it's center.

The script can call objects by name but you need to drag and drop that object into the gui's script section of the object to associate them properly.

A smoothing function is needed - the analog input is not a smooth function, so the finger movement looks very jerky with raw data.

You can map the analog output to the correct angle in the Arduino code by using the map(xi, yi, xo, yo) function.

Here is the one fingered Arduino code that I used. Notice that I'm using a 50 ms delay between each loop. I found that this help with the jitteriness that I was seeing. I'll get rid of this later when I add a smoothing function to my script.

Next I added a second digit. Here I found that my code using eulerAngles didn't work very well anymore. That's because the second digit was still rotating around the origin intead of rotating around the joint. With Amira's help, I used the RotateAround function instead offsetting the 'origin' of the second joint to the end of the first digit. I still need to implement a smoothing function since the analog input is still quite jittery but here is the code thus far:

And here is a short video of the code in action!

The Source Code

C# code for Unity

Arduino code for index finger only