Lily Yeazell

I want to make a magic marker that can draw on any flat surface. I'm thinking of trying to use an IMU with 9 degrees of freedom to keep track of the pens location. Then I'll have some sort of application interface where you can see what is being drawn.

There exist similar products with a drawing pad where you draw on the pad and it shows up on your computer, but the main difference is that I want it to work on any surface.

Basic overview of system

Design Considerations

• Microcontroller: The main challenge I anticipate is speed especially with regard to the imu. I think having a dual core microprocessor could be useful. Position tracking is challenging so my plan is to have one core just running IMU operations. That way it won't drift due to lags. Then the second core can be used for the bluetooth communication, and just read the current position from shared memory. One option for this is the ESP32S3.
• Communication: I need to be able to communicate between my device and my user interface. The two main options are bluetooth or WiFi. I'm leaning towards using bluetooth because I tried to do some WiFi stuff in input devices week and found it a bit cumbersome. The quality of WiFi is really variable and getting the device to connect sometimes took a bit. The variability also made it hard to debug. I am interested in exploring bluetooth in javascript so I will see how that goes. I think in Networking and Communication week, I'll start by trying to work with bluetooth and pivot to wifi if that fails
• Power: I don't want to have to use a wire to power my device, so I need some sort of in device power. When I get to the packaging phase of my project I will need to do some power calculations to figure out what sort of battery I need
• User Interface: I'm pretty familiar with HTML and javascript so I think I'll stick with that. But maybe worth considering alternatives *shrug*

Tasks

• Read IMU Data: connecting an imu to my microcontroller and integrating over acceleration to obtain a rough position
• Transmit IMU Data: using bluetooth to send the marker position to a web ui using bluetooth. This will be my communications and networking week project
• Basic Interface: A web UI to display a drawing based on device position. As they move the marker the drawing will accumulate. At this point I will assume the user is drawing on horizontal plane and choose an arbitrary orientation.
• Choose Drawing Plane: The user can lie the marker along a plane and press a button to indicate this is the plane they want to draw on. Then the user can tap four points with the marker to indicate where they want to corners of their surface to be. Will require some small changes to both how position is computed and how the UI decides canvas size
• UI Improvements: There are a lot of UI improvements I can foresee making. Some of these I might be able to do while working on the basic ui and others might never get done. Some Ideas: changing marker color or size, erasing, drawing quality, etc.
• Packaging: below is a rough idea of what I see the packaging looking like. I'll have to figure out on device power. I will also need to make decisions about how to fit it in as small a space as possible and how to include an antennae for bluetooth.
• Final Touches: Leaving myself time here to make whatever changes I didn't have time for.
• Video and Presentation: I need to make a 1 minute video explaining my project

idea for what the packaging will look like

Updates

• (12/4): the imu is not working as well as I'd hoped. I will play around with it for a bit more but if I can't get it more accurate, I may have to make some changes to my project. I think I could pivot to a camera based led tracking system for drawing, where you would have a camera module and a marker module (with an led on the end) and the same drawing website. Might look into ir leds as well.