Objective
I’ve been experimenting heavily with the ESP32S3 Sense in the past weeks, primarily because I believe it could provide some additional utility for my final project because Apple Watches do not have cameras. Things like QR code menus have made this a necessity in day-to-day life.
So, this week, I wanted to try to make this functionality a reality by programming the ESP32S3 to read QR codes.
Approach
Initially, I planned to use a QR code reading library directly with the ESP32S3, but experienced some challenges with the limited memory on the microcontroller. So instead I decided to offload the heavy lifting and found a free QR code reading API through Circuit Digest (https://www.circuitdigest.cloud/). Instead of processing the QR code on the microcontroller, I would take a picture of the QR code with the ESP32S3 Sense’s camera and then send that image to Circuit Digest’s server to decode it.
Backend
I stuck with Arduino IDE for the coding part of this project and used the CameraWebServer example as a basis to get the camera initialized properly.
Next, I followed insturctions from Circuit Digest to get an API key and how to structure interactions with their server. Thankfully, they had an example using an ESP32 CAM, so it was mainly a matter of modifying these.
However, they used external switches to capture the photos and an attached OLED screen. For this week I wanted to do everything through a webapp, so this required some modification.
To keep things simple, I decided to get the QR code working first in the Serial Monitor before adding a Front End.
First, I needed to make sure messages were printed in the Serial Monitor rather as opposed to their external OLED screen. I also needed to change the trigger for capturing the image and sending it to be a serial monitor command instead of a switch press.
After a few tries, I had the functionality working to take my own picture. Next, I needed a QR code! I quickly made one to navigate to my HTMAA site:
Next, I lined up the photo with the live stream of the camera and then entered the ‘CAPTURE’ command in the Serial Monitor. This took a couple tries because I didn’t have both visible at the same time. Additionally, early on, for some reason the microcontroller would only accept a command in the first ~10-15sec of being hooked up.
Still, eventually I had success!
Frontend
For the front end, I again wanted to start simple. I added some basic html into my code to incorporate the camera view, a capture button, and messages from the Serial Monitor into a simple webapp:
Integrating the backend and front end proved more difficult than I expected, though. The microcontroller was still giving me problems where it wouldn’t take commands after the first few seconds of being connected. This made it really difficult to get the front end open and aimed at a QR code in time to take a picture.
My solution ended up being to add some additional lines of code to have it continue to search for a serial input; it seems like initially it would run the command to look for an input, then time out and never restart.
After I added this fix, the button worked, but messages wouldn’t populate on the front end until the whole process was completed (ie they weren’t real time). Otherwise, though, it worked!
A success is a success.
Files: