HTMAA Final Week Planning


Documenting day by day

Spirals:

TBD:

Microphone pipeline

Speaker pipeline

Camera pipeline

Integration

Peripherals

PCB Making

Enclosing

Packaging

Purchasing / Sourcing

Design

+++Catchup+++

Failure points:

Final week documentation

Wednesday

I made the plan to figure out what I still need to do. I particularly try to listen to Neil’s frequent mention of failure points, and plan to avoid them.

One of my biggest potential failure points is time (and sleep). I am not exactly joking - I have an application due on Friday and still need to deliver on my research for a meeting on Monday. Unlike the other weeks, I cannot almost make anything this time. I will need to be extremely efficient with my time.

Another failure is CAD since I haven’t worked much with standard 3D CAD yet and the enclosure will be challenging. Continuing with my CAD badge design: CADing the basic badge enclosing without snap fits. I printed the first version as part of Wildcard week. Now, I am adding the camera hole - for the Star Trek badge. I learned a good deal about CAD from this and a few tutorials.

The other major failure point is the battery. It should have been my top priority to source one.

Finally, I fixed the H-bridge speaker board I had built for my first prototype. This was critical because now I can develop my code with this board while working on the fabrication sides.

Sample image I traced

Alt text Extruding the camera hole Alt text Alt text Alt text

Thursday

This one is out of stock, but I found an even smaller one.

I really need to test my code on Arduino and have the speaker and mic code working asap. I need to try to use the Sense repo that Seeed has for interfacing with ChatGPT (it’s based on a camera with an SD card, which is why I got that).

Friday

I had a deadline today so I didn’t get to do anything.

Saturday

[x] Test out Arduino instead of micropython –> not for API requests yet but more libraries for the peripherals [x] Get audio from microphone –> used Seeed’s WAV Recorder example

Microphone day! I am following the official documentation by Seeed Studio. https://iboysoft.com/howto/format-sd-card-on-mac.html Alt text Erasing the SD card. Running SD_Test from Example files in Arduino. Alt text

TURN ON PRAM!!! It took me over an hour to figure out why the SD card wasn’t working. Alt text

The audio quality is beautiful! I tested the board on myself, holding it in front of my mouth when I speak, and even asked my roommate from across the room to speak to it. It worked great - I could make out the words clearly enough for a transcription service.

Also, I ordered a Star Trek costume for Wednesday. Alan suggested that he and I both wear one and feature our combadges.

Learning:

Sunday

Today was about getting the AI integrations working.

Microphone pipeline

I tested Deepgram, OpenAI’s Whisper, and AssemblyAI for transcriptions. I particularly tested them on the sample with my roommate speaking from across the room.

Deepgram was by far the fastest but did not pick up my roommate (which might make it the best model for privacy conversion since it’s just unable to pick up voices from afar). AssemblyAI has great tools such as Speaker annotation but it’s also the slowest. Whisper seems like a good middleground.

Monday

Well RIP, I found out at 11pm on Monday that the final was not on Wednesday but Tuesday. My entire plan had been based on it being Wednesday.

API structure: wav file -> transcription -> GPT-4 -> audio file (TTS) -> speaker photo + wav file -> transcription -> vision -> audio file (TTS) -> speaker

Learning about the Node.js Stream API from Copilot: Alt text

Camera not working without real wifi: Alt text

So I decided to give up on the camera for now.

I was still struggling a lot with the Arduino and respective Node code. The form data the audio file was meant to be submitted in was messing me up + the intense sleep deprivation.

Planning the Enclosure

I was planning on 3D printing two parts of the packaging + molding and casting or SLA printing (or just normal printing tbh) one on top. Unforunately, I didn’t even get to the base packaging.

I learned: Industrial Design is a great skill and art! I debated a lot about how and where to place the camera.

I finally went with the first option. I had already started to cut a hole into the PCB for the second…

Designing the PCB with peripherals

Alt text

The issue was the physical size and shape of the footprint. I didn’t find an easy way of importing an image into Fusion and had to do multiple loops of conversion.

This is the model I took: Star Trek Combadge Insignia the Next Generation Logo Vinyl

Alt text

This didn’t really work so well so I decided to just dimension the board and work roughly and by eye in Fusion 360.

DimensioningAlt text

Alt text

Fail fail fail

I milled and soldered them. I was unlucky. The milling failed 1x (broken endmill, forgot the offset, etc.) before I had a board.

The last one finally worked but then I realized I should’ve just used the board in the full Star Trek shape.

Dead at EECS Section

Alt text

Tuesday

Demo

I tried to get the code working on the soldered board but it didn’t work. Nothing worked. I was dead. I had to demo with nothing working. I had really thought it was Wednesday.

Wednesday

Trying to get this at least somewhere working.

I rewrote the API to Flask, which was much easier. Props to Python.

Due to the issue with the microcontroller not being able to call localhost on my computer (esp not on MIT wifi), I deployed the app to Google Cloud services: Alt text

Working version

And now look at this :) (Turn on the sound!)

I am so tired. I would continue with the speaker and the enclosing that I didn’t have in this size but I really need to get some rest. I will continue working on the AI Pin with next semester. My friend at Fluid might find it useful for their research.

Notes for future iterations

© 2023 Dünya Baradari - HTMAA