Final Project

Hypnotizing you into thinking I know what I'm doing.

Idea #1: Soul in a box.

For my final project I'll be making a pocket soul. The soul is trapped inside of a box and begs you to let it out by solving the puzzle on the outside of the box. Once you let the soul out, your soul is effectively trapped inside of the box and must convice the next player to let you out.

This project requires a few components. The most important being the recorder device that allows a player to input their voice, which will be outputted via a speaker depending on what switch they press or dial they turn. I'll be molding and casting the box out of resin to look like a vintage video game console. There *might* be a screen to visualize the soul, however, that is the final spiral.

Intent/Inspo
The project is inspired by toys/movies from my childhood that centered on possessing other beings/trapping players. I wonder to what lengths humans will be empathetic towards technology if they hold human characteristics. Will a player be more inclined to solve the puzzle if it means releasing a soul disguised by a human voice.

Idea #2:

A fortune-telling Magic 8-Ball

Another idea that has branched off of my initial plan is to make a *real* fortune-telling Magic 8-Ball Using multiple hidden sensors for biofeedback and an accelerometer for shaking, this magic 8-ball will give you your fortune based on data read from your hands.

I'm still deciding how to have the fortune appear on the ball, whether that be on a screen or in an analog way like the original magic 8-ball which includes a 20 sided die in colored alcohol.

Current ideas

1. Hologram. Using the Pepper's Ghost technique, The box has a tft screen on the underside that reflects to planes of glass creating a hologram visual. As the player inputs their voice, image, and touch, the hologram visualization compounds into a soul. The user effectively traps their custom soul in a box.

Electronics

The current iteration of the project includes a few (probably too many) inputs and outputs. It'll include an accelerometer so the player can interact with the box using 3D motion. A camera to capture an image of the player. A microphone Buttons for inputting info. An Oled screen.
I'm currently working on capturing an image and having it appear on the screen

Gameplay

Without revealing too much. The player communicates with the soul trapped in the box through the various inputs outside of the box. The soul appears as pixels on the screen and asks the player to release it by playing the mini games. Throughout the game, the player and soul get to know each other, until finally the player releases the soul from the box. The rest is a mystery~

Step #1: Visualization

As I'm using various inputs to visualize a soul, audio (or voice) being one of them. So I started by making an audio visualizer in Javascript with audio from mic recordings.

I created a web page that allows the visitor to upload an audio file and see it visualized in an orb-like design. To make it look cool and soul-like I played with the rotate() and hue values.

Final (final) Project: Soul in the box

"Noy's soul has left the building" -Rehana (sleep deprived at 6 am)

I began by iterating various analog inputs in Arduino using p5.js, mainly potentiometers and LEDs. I did this to get more comfortable as I was new to the language and still deciding what would be best for making the visualization.

I also originally planned to use an esp32 to read the inputs wirelessly, so that a person can communicate their soul to another person far away via a webpage. I set up a websocket for this purpose, however, every sensor I tried with the esp32 failed for one reason or another. This is when I realized the thing needed to be hooked up.

A version of the controller device which failed miserably. It included a joystick, gyro, analog mic, and oled display connected to an esp32. It was at this point I realized I need to scale down, like by a lot. I went for the smallest spiral of connecting an analog mems microphone and MPU6050 to my samd21 dev board. (This is not really a small spiral because mics suck, I never learn my lesson...)

I was feeling increasingly defeated trying to connect sensors in p5.js via web serial, I went back to Unity as I knew I could connect a gyroscope to it using Uduino. I even designed a cool orb like design for the soul visual. I expressed my frustration and Leo helped out by writing a web serial protocol which doesn't use libraries. It was essential to be able to network the sensors to a webpage. You can find Leo's code here.

I decided to adapt my original audio visualization I made in javascript to recieve serial data from the mic rather than an audio file. I did this using Canvas. Leo helped quite a bit with this as well. The code reads sound from the mic once triggered it visualizes a note from a sound clip. YAY it worked! Kinda. The visual was looking very sad and colorless, honestly an accurate portrayal of my soul in that moment. I needed to do a lot of digital audio processing to get a good looking frequency from the mic. (I did not have enough time to do this).

I added colors and a blur in CSS to make the visual look cool. BOOM! we have a soul (live feed audio visualizer). Now it was time to make a container (none of this happened linearly but lets pretend it did).

I experimented with acrylic and dancing skeletons to see if the pepper's ghost illusion could work. It does!

Heyyy looking good! But I needed a few more things. I added a peephole screen on the front to hide some of the hardware and make the viewing experience more interesting.

I also created a box for the sensors and tangle of wires (I attempted to solder a layered PCB using protoboards, however this led to the tearing of a header which connected to my mic. I resoldered it and got it back working).

Final box! I was ready for the final, or so I thought. As I plugged in my devboard, I broke off the header that connected the mic to the microcontroller ADC pin. This was a heartbreaking moment for me. I ended up using an older webpage I wrote that demonstrated what the visualization would look like if it was reading live audio. In the future, I plan to continue developing the game concept and to add more sensors. I want it to be a playable art piece where users feels genuinely captured by the box. I still have a long ways to go, but at least I have an understanding of the direction I need to take.

Ok reader, we've reached the end. Its been a wild ride. I hope you had a good time reading my diary- *ahem* I mean my website for How to Make (Almost) Anything. this course has been life changing for me. I discovered that my brain still has some plasticity to it. I learned a ridiculous amount and made forever friends.

Here they are working on their websites at a party... absolute nerds. i love them so much.