// Cambridge, MA

week 14: wild card week

for wild card week, i decided to explore embedded AI.

as a refresher, my final project is MOOD LIGHTING, an LED lighting array that changes colors depending on how you feel. originally, i had planned to run the ML model on my computer and just make a device that changes color, but this week seemed like a good opportunity to try to do all of the modeling on a chip.

i found a high-quality dataset for emotion recognition with 32,000 48x48 images classified as either neutral, happy, sad, disgust, fear, surprise, or angry (see below for angry).

fine tuning a CNN in python using Keras got me as high as a 65% validation accuracy.

i decided to use Edge Impulse to make a tinyML model that i could use on a board. unfortunately, using their default settings got me a validation accuracy below 40%:

i went back and fiddled with parameters, and learned that it performed best when i just made it a binary classifier between happy and sad using the MobileNetV2 architecture; validaiton accuracy was about 72%.

although i'm unsure if i want my LED to always be happy or sad--a neutral state would be useful for lighting design, so i trained my final model to have three classes at the cost of a bit of accuracy.

i have my model as a ZIP file ready to be uploaded to my board, but...

unfortunately, i only got my esp-32 over the weekend and had all sorts of trouble with it.

i got the board to flash blinking code correctly, so i'm confident that it works. but whenever i try to set up a wifi connection, i see a variety of issues; first, the Harvard default wifi didn't work because of authenticaton issues. i tried my home wifi at 5 GHz and 2.4, and neither network worked. i kept seeing some version of this error:

i also tried to set my computer up as a 2.4 GHz wifi hotspot per sophie's suggestion, only to find my computer does not have this ability :(

luckily, i connected with Erik yesterday and he was able to get an esp32 with camera working on his super-secret CBA network at the Media Lab. so i'm going to come by this week to debug, and worst case, use his board.

--

for an update on progress from this week, see the final project page!