Week 9: Input devices

Materials

Idea

This week is heavily related to my final project, in that I want to start creating a device that can control the pan and tilt of a dual axis mirror using computer vision. To put it more simply, the input here is a set of coordinates that are computed using facial detection software. These coordinates will then be sent to LEDs and stepper motors in next week’s assignment on output devices.

Process

Going in line with past learnings I’ve had working with electronics, I started this whole process by listing out everything I needed my device to do— this included inputs, outputs, and any microcontrollers I would need along the way to communicate between the peripherals. This process resulted in the diagram on the left.

Once that was done, I was able to identify which components in this device were serving as the "inputs", pictured on the right.

At first, I was pretty sure I wanted to use the OV2640, which is a camera module that can attach to ESP32S3’s for simple camera applications like video streaming and low level image processing (pictured below).

Amazon fucked up and sent me the wrong product entirely, so I didn’t get this camera module at all this week. During the time I spent waiting for my camera module to arrive, I spent a lot of time exploring other microcontroller and development kits. What I found during this process is that facial detection and recognition are incredibly expensive computing tasks for any microcontroller to handle.

For my final project, I want to make sure whatever I end up creating can handle image processing on-chip (i.e. without Wifi/cloud connection). I knew going forward I’d be fairly limited on what I could use, so I ended up opening an issue on the class gitlab and the ESP32S3-EYE development kit was recommended as a cheaper way to handle on-chip image processing at a moderate rate with a pre-built facial detection library.

While waiting for my ESP32S3-EYE development kit to arrive, I started working on the rest of the input function: converting X and Y-axis analog data to instructions for my stepper motors and LEDs to intake. That way, I can plug and play my camera straight into my circuit when it arrives next week. In order to do this, I used a joystick that computes an analog signal whenever moved in X and Y directions, then converted that data into logic the stepper motors would be able to understand.

My first error occurred when I got ahead of myself and tried to hook up inputs and outputs at the same time without validating I had a functioning circuit component by component. Without realizing it, I had sent current from both my computer’s USB-C port and an external 5V power supply, which resulted in me frying my Arduino. After learning from my mistake, I disconnected my power supply until I would need it later and used just my USB-C power.

My second error was that I didn’t connect my microcontroller and attached joystick to a common ground, which resulted in highly variable and flawed sensing data. On a previous personal programming project, I had the same phenomenon occur where my LEDs were randomly flickering, regardless of how I programmed it, so I knew fairly quickly how to diagnose this issue. Once that issue was resolved, I was able to read analog input from a joystick and convert it into coordinates that my LED strip and stepper motors will be able to understand next week when we work on output devices.

To hook my analog sensor up with my RP2040, I milled a new development board (traces and outlines here) that had all the parts I’d need for this week, Output Devices week, and Application/Interface Programming week.

I created the following sketch to control my peripherals.