Project Overview
au5mented piano is a playful augmentation of MIT Sigma Alpha Epsilon's grand piano, combining mechanics, electronics, and software to enhance its capabilities:
- Piano-playing automaton: A solenoid-driven contraption wrapping over the keys to play the piano autonomously.
- Real-time MIDI over WiFi: An ESP32-C3 network drives the solenoids based on MIDI input streamed from a Flask-hosted web interface.
- Volumetric Display: A parallel project uses a spinning 64x64 LED panel for a persistence-of-vision (POV) effect.
Keyboard Kover
I designed and milled a PCB to drive 5 12V solenoids using N-MOSFETs. Here’s the schematic for the solenoid driver board:
The PCBs were soldered and mounted onto the OSB "Keyboard Kover" I created in Week 10:
To reduce noise and improve key actuation, I designed TPU tips to attach to the solenoids:
WiFi MIDI Communication
For communication, I implemented a server-client system over WiFi:
- The server is a Flask-based web interface for selecting MIDI files and setting the tempo.
- ESP32-C3 microcontrollers receive MIDI messages and activate solenoids based on assigned note ranges.
Here’s a demonstration of the real-time MIDI streaming:
Below is a snapshot of the Flask-based MIDI file selection UI:
Challenges & Solutions
- Power issues: Some ESP32-C3 boards were overdriven with current, rendering them unserviceable. This limited my demo to a single board controlling 5 solenoids.
- Timing drift: Real-time MIDI streaming introduced slight note drift. While notes play correctly, they are not perfectly evenly spaced.
Despite these challenges, the system successfully demonstrates the core functionality and scalability for future iterations.
Volumetric Display: Parallel Project
In parallel, I developed a volumetric display capable of rendering 3D OBJ assets. Using a brushed motor, a spinning 64x64 LED matrix achieves a convincing persistence-of-vision effect. The display is synchronized using an IR break beam sensor and controlled by an FPGA (CMOD A7-35T).
Final Demo
Here’s the final demonstration of the au5mented piano:
Next Steps
- Resolve power supply issues to scale up the solenoid driver system for 88 keys.
- Refine real-time streaming to eliminate note timing drift.
- Integrate the LED panel for note animations and develop a light show synchronized with live performances.
← Back to Main Schedule