Final Project

👉 HTMAA Final Week Documentation

Final Idea: AI Pin

The key to really increasing the usability of AI for humans is context.

Understanding context is hard. We need to keep track of what is going on around and within a person.

Largely to that end, a couple of companies are currently releasing AI pins and related wearables.

Not sure what this can do for 699$ + 25$/m, what ChatGPT API can't do inside a app or some 5$ ESP microcontroller and a 1$ cam...
Cool idea but i don't get the pricing...

— David Krammer (@davidkrammer_) November 9, 2023

I just built the world’s most personal wearable AI!

You can talk to Tab about anything in your life. Our computers are now our creative partners! pic.twitter.com/RraiOPL4sm

— Avi (@AviSchiffmann) October 1, 2023

So many of the key hardware features of these pins are also present in the Xiao ESP32-E3 Sense. I feel challenged to make my own DIY cheap AI pin with it.

Inspiration: Star Trek Combadge

Much of my inspiration for the design still comes from my Idea #2, the Star Trek pin, which will be very similar (just with some AI capabilities).

The Combadge sells on Amazon for $65. Anyone I tell about this think that’s crazy. But it is done neatly - it’s very light and small. So I ordered it as a model.

It also has a really nice overview of the features:

The way the Combadge works is that it has a microphone and a speaker. It listens for the “Hey, Siri” keyword and then sends the audio to the phone to be processed. The phone then sends the response back to the Combadge, which plays it through the speaker.

More inspiration: Walnut Speaker

There is amazing tiny walnut speaker by Penguin DIY.

How we go about this

Core:

• conversation with ChatGPT
• need microphone (on Sense) to listen to spoken words, send them to
• Chip runs micropython
• needs its own custom functions to do activities, like using the camera -> that would trigger a photo to be sent via API to OpenAI Vision API

Nice to haves:

• Cellular connection -> to make calls and have mobile data
  • an eSIM would be nice but for that, apparently, you need a special deal with the network providers (don’t think I’ll get to that in a month), so instead, we’ll have to go with a regular SIM
  • SparkFruit has a great cellular module: MicroMod Cellular Function BoardKit and a starter kit
• Name recognition for privacy reasons (the pin gets a name and recognizes when it’s called, like “Hey, Siri”) -> TinyML, there are a few tutorials
  • with this, the pin would only send out information to external APIs when called and instructed, so to not share more data with the big companies :)

Integrating the Weeks

My input and output weeks are related.

In the input week, I tested the camera and in the output week the speaker.

I realized I could just use the H-bridge board for my speaker, which lead to my first prototype.

In networking week, I connected my board to an external API via urequests.

Design

Iron Man Arc Reactor Heart

Previous Ideas