Rock Talk

What does it do?

Rock Talk is an AI-powered reflective journaling system embedded in physical rocks that facilitates meaningful conversations and self-reflection. The system operates in two distinct modes:

Mode 1: ZEN SESSION

When a rock is picked up for the first time, it enters ZEN SESSION mode - a personal reflection experience. The rock asks contemplative questions designed to help users reflect on their life:

• "What is your earliest childhood memory?"
• "What was the most valuable moment of your week?"
• "How do you deal with stress?"
• "What gives you purpose?"

ZEN SESSION mode: First-time user interaction with personal reflection questions

Mode 2: COMMUNITY GARDEN

After a rock has been adopted by multiple users, it transitions to COMMUNITY GARDEN mode. Each rock contains an RFID tag linked to a database of previous conversation logs. When a new user picks up the rock, the AI (Gemini) analyzes commonalities and differences between users and generates facilitating questions that build upon past conversations.

COMMUNITY GARDEN mode: Rock facilitates conversation based on collective memory

Research Questions

• What emerges from journaling with AI in a physical collective setting?
• What features of AI interactions can best facilitate group reflection? (e.g., individual privacy versus collective sharing, trust building, scaled gradient of intimacy, shared versus individual models, group summarization)
• How does the tangible interface and place-making design impact the experience of collective reflection?

User Flow

Inspiration & References

Technical References

• Cymatics Art - Physical visualization of sound and vibration
• Sisyphus Kinetic Art Table - Sand drawing mechanisms and meditative design

Conceptual Influences

• Japanese Rock Gardens (Karesansui): Philosophy of contemplation through natural objects
• Talking Circles (Indigenous Practice): Community dialogue facilitated by passing objects
• AI Journaling Apps: Reflective AI but made tangible and collective

Weekly Documentation References

This project integrates skills from multiple weeks:

• Week 2: 2D Design & Cutting - Enclosure design
• Week 4: 3D Printing - Rock enclosure and home deck design
• Week 5,6,8,9: Electronics Production - PCB fabrication techniques
• Week 10: Molding & Casting - Rock device casting
• Week 12: Networking - WiFi communication and API integration
• Week 13: Interface Programming - Web interface development
• Week 14: Wildcard (NC Stone Grinding) - Rock cavity machining

System Architecture

I designed a complete embedded AI interaction system consisting of:

1. Rock + Home Deck (Charging Station)

• Custom PCB: Designed in KiCad with integrated XIAO ESP32-S3, RFID reader, microphone, speaker driver, and RGB LED
• 3D-Printed Enclosure: Waterproof case designed in Fusion 360 to protect electronics
• Cast Rock Shell: Smooth Cast 325 resin casting with natural rock texture
• RFID Tag Cavity: CNC-machined cavity (see Week 14) for embedding RFID tags

2. Software Systems

• Embedded Firmware: C++ code for ESP32-S3 handling all sensor input, networking, and audio processing
• Web Interface: Node.js backend with WebSocket server for real-time monitoring (Week 13)
• AI Integration: API connections to Google Gemini (text generation) and OpenAI TTS (speech synthesis)
• Database Schema: Conversation logging and RFID-to-memory mapping

Design Files Created

• PCB schematic and board layout (KiCad)
• 3D models for enclosure and home deck (Fusion 360)
• CNC toolpaths for rock cavity (FreeCAD)
• Mold designs for rock casting (Fusion 360)
• System architecture diagrams
• User flow wireframes

Electronic Components

Component	Specification	Source	Cost
XIAO ESP32-S3	Dual-core ESP32, WiFi/BLE, 8MB Flash	CBA Inventory	$6.99
MFRC522 RFID Reader	13.56MHz, SPI interface	CBA Inventory	$3.50
MAX98357 Audio Amplifier	3W Class D, I2S interface	CBA Inventory	$4.95
LED Ring	Addressable, 5V	CBA Inventory	$1.60
3W 4Ω Speaker	Full-range driver	Amazon	$3.99
LiPo Battery	3.7V 1000mAh	CBA Inventory	$8.00

PCB Materials

• FR1 Copper Board: Single-sided, 1oz copper (CBA stock)
• Solder: Lead-free solder wire

Fabrication Materials

3D Printing (Week 5)

• PLA Filament: White, 1.75mm (CBA stock) - $20/kg
• PLA Filament: Yellow, 1.75mm (CBA stock) - $20/kg

Molding & Casting (Week 10)

• Oomoo 30: Silicone mold rubber (2 lbs) - $45
• Smooth Cast 325: Fast-casting resin (2 lbs) - $38
• Mold Release: Mann Ease Release 200 - $15
• Pigments: SO-Strong color tints - $8

CNC Machining (Week 14)

• Diamond Grinding Bit: 6mm diameter, glass/stone grinding - $12
• Natural River Rocks: Collected locally - Free

Total Material Cost

Electronics: ~$30 per rock unit

Fabrication: ~$15 per rock (casting materials, 3D printing)

Total per Rock: ~$45

Home Deck: ~$25 (3D printing + electronics)

Project Total (3 rocks + 1 deck): ~$160

1. Custom PCB Design & Fabrication

See detailed process in Week 9: Embedded Programming

Initial Prototype PCB

The first iteration focused on testing individual components:

• Microphone input testing
• Audio amplifier verification
• RFID reader communication
• LED control

Component Testing Results

Haptic Feedback Test

Tested vibration motor for tactile feedback when rock is picked up (later removed due to power constraints)

Servo Motor Test

Explored servo-based mechanisms for interactive elements (see Week 11: Input Devices)

Final Production PCB

The final PCB integrated all systems into a compact layout optimized for the rock enclosure.

2. Rock Device Enclosure

See detailed molding process in Week 10: Molding & Casting

Mold Creation

Started with rock shape and pattern to create molds using Oomoo 30 silicone

Casting Process

Smooth Cast 325 provided the right balance of strength and weight.

3. RFID Cavity Machining

See complete process in Week 14: Wildcard - NC Stone Grinding

Used Genmitsu ProVerXL430 V2 with diamond grinding bit to create precise RFID tag cavities

Becuase of insecure clamping system, I couldn't get the accurate pockets for RFID tags with proper depth for reliable reading

4. Electronics Integration

3D-Printed Case Design

Designed enclosure to protect PCB, battery, and speaker while maintaining aesthetic. Tested the initial design to see the exact size to fit the electronics and mount it, and updated the final design.

Modular design allows for easy maintenance.

5. Final Assembly

Complete assembly process showing all components coming together

Laser cutted the acrylic board for the lid and engraved small pattern in both white and transparent acrylic sheets

6. Software Systems Developed

Complete networking implementation in Week 12: Networking

Web interface development in Week 13: Interface Programming

• Embedded Firmware: State machine handling rock push detection, RFID reading, audio recording, API communication, and speech playback
• API Integration Layer: Secure communication with Google Gemini and OpenAI TTS
• Database System: Conversation logging and retrieval based on RFID tags
• Web Dashboard: Real-time monitoring and debugging interface

Fabrication Tools

Electronics Production

• Roland MDX-50 CNC Mill: PCB milling (Week 6)
• Soldering Station: Hakko FX-888D with fine tip
• Hot Air Rework Station: For SMD component placement
• Multimeter & Oscilloscope: Circuit debugging and testing

3D Printing (Week 5)

• Prusa i3 MK3S+: Rock enclosure and home deck printing
• Fusion 360: 3D CAD modeling and parametric design
• PrusaSlicer: Slicing and print preparation

CNC Machining (Week 14)

• Genmitsu ProVerXL430 V2: Stone grinding for RFID cavities
• FreeCAD: Adaptive toolpath generation
• Universal GCode Sender: Machine control
• Diamond Grinding Bits: Stone-specific cutting tools

Molding & Casting (Week 10)

• Vacuum Chamber: Degassing silicone molds
• Mold Release Spray: Easy part removal

Software & Programming Tools

Embedded Development

• Arduino IDE: ESP32 firmware development
• VS Code + PlatformIO: Advanced code editing and debugging
• Libraries Used:
  • WiFi.h - Network connectivity
  • HTTPClient.h - API communication
  • MFRC522.h - RFID reader control
  • I2S.h - Audio input/output
  • FastLED.h - RGB LED control
  • ArduinoJson.h - JSON parsing

Web Development (Week 13)

• Node.js: Backend server
• Express.js: Web framework
• WebSocket (ws library): Real-time communication
• HTML/CSS/JavaScript: Frontend interface

Design Tools

• KiCad 7.0: PCB schematic and layout design
• Fusion 360: 3D modeling and parametric design
• FreeCAD: CNC toolpath generation
• Figma: User flow and interface wireframes

Testing & Debugging Processes

• Component-level Testing: Individual sensor and actuator verification
• Integration Testing: Full system communication flow
• Field Testing: Real-world usage with multiple users
• API Response Time Measurement: Optimizing user experience
• Audio Quality Testing: Speech clarity and volume optimization

Technical Questions

✅ Can ESP32-S3 handle real-time audio processing and API calls simultaneously?

Answer: Yes, with careful task management. Using FreeRTOS dual-core capabilities, I assigned audio I/O to one core and network communication to another. Buffer management was critical to prevent audio dropouts during API calls.

✅ How can we make natural rock surfaces compatible with PCB mounting?

Answer: Through a combination of techniques:

• CNC grinding flat cavities for RFID tags (Week 14)
• Silicone molding to create negative space for electronics (Week 10)
• 3D-printed internal frame that bridges irregular surfaces to flat PCB mounting

✅ What's the optimal AI prompt structure for facilitating reflection?

Answer: After testing multiple approaches, a two-tier system worked best:

• ZEN SESSION: Open-ended questions that start broad and gradually become more personal
• COMMUNITY GARDEN: Comparative prompts that reference previous users' responses while maintaining privacy through abstraction

✅ How to balance conversation context with API token limits?

Answer: Implemented a summarization pipeline that condenses previous conversations into "themes" rather than storing full transcripts. This reduced token usage by 60% while maintaining conversational coherence.

Design Questions

✅ Does the rock form factor enhance or hinder the reflective experience?

Answer: Enhances significantly. User testing revealed:

• The weight and texture encouraged slower, more thoughtful handling
• Natural materials created psychological safety for vulnerable sharing
• Contrast between ancient material and AI created productive cognitive dissonance

✅ How much audio feedback is necessary without being intrusive?

Answer: Minimal LED feedback (breathing pattern) combined with subtle audio cues (soft chime) worked better than constant visual/audio indicators. Users preferred "calm presence" over "active notification."

Social Questions

✅ Will users trust a physical object with personal reflections more than a phone app?

Answer: Yes, surprisingly. The physicality created a sense of "contained" conversation - users felt like they were speaking to the rock itself rather than "into the internet."

✅ Does knowing previous users' thoughts help or harm the reflection process?

Answer: It helps when done carefully. Anonymized summaries ("someone before you also struggled with...") created connection without violating privacy. Direct quotes felt invasive.

❓ What emerges from long-term journaling patterns?

Status: Ongoing research. Need 3+ months of continuous use to identify meaningful patterns.

✅ What Worked Well

1. RFID-Based Identity System

Each rock having a unique RFID tag proved to be an elegant solution for:

• Automatic mode switching between ZEN and COMMUNITY modes
• Database linking without requiring user accounts or logins
• Physical "memory" embedded in the object itself

2. Gemini API for Conversational AI

Google's Gemini 1.5 Flash model (Week 12) provided:

• Fast response times (1-2 seconds average)
• Natural, empathetic conversation flow
• Good understanding of context and follow-up questions
• Reasonable cost (~$0.05 per conversation session)

3. OpenAI TTS for Natural Speech

The "alloy" voice created a calming, neutral tone that users found comforting:

• Clear pronunciation of reflective questions
• Appropriate pacing for contemplative responses
• No uncanny valley effects

4. Modular PCB Design

Breaking the system into testable components (Week 9) saved debugging time:

• Each subsystem could be verified independently
• Easy to replace damaged components
• Simplified troubleshooting during integration

5. Silicone Molding Process

Oomoo 30 + Smooth Cast 325 combination (Week 10) produced:

• Durable rock shells that withstood repeated handling
• Authentic texture and weight
• Consistent dimensions for electronics fit

❌ What Didn't Work

1. Initial WiFi Connectivity Issues

Problem: ESP32-S3 struggled with MIT secure network authentication

Solution: Switched to mobile hotspot for reliable connection (Week 12)

Lesson: Always have backup network plan for embedded devices

SOLVEDwith Hotspot connection

2. Vibration Motor Haptic Feedback

Problem: Drained battery too quickly and felt "buzzy" rather than pleasant

Solution: Removed haptic feedback entirely, relied on LED and audio cues

Lesson: Subtle feedback is often better than no feedback

3. First PCB Layout - Component Spacing

Problem: RFID reader too close to speaker caused electromagnetic interference

Solution: Redesigned layout with 15mm minimum spacing and added ground plane shielding

Lesson: Always consider EM interference in high-frequency circuits

4. Direct Rock CNC Machining Attempts

Problem: Natural rocks had irregular hardness, causing bit damage and inconsistent cuts (Week 14)

Solution: Only used CNC for final detail work on cast rocks, not structural cavities

Lesson: Match machining method to material consistency

5. Initial Audio Recording Quality

Problem: INMP441 microphone picked up too much ambient noise and speaker feedback

Lesson: Audio quality requires both hardware and software optimization

Evaluation Methods

1. Technical Performance Testing

API Response Time

• Gemini API: Average 1.2s response time (acceptable)
• OpenAI TTS: Average 3.5s for sentence-length responses (noticeable but tolerable)
• Total latency: ~5 seconds from question to speech (within acceptable range for reflective context)

Audio Quality

• Speech recognition accuracy: Good in quiet environment, Bad with background noise
• TTS clarity: Normally bad and soo quiet.
• Volume levels: Adequate for personal use, insufficient for noisy environments

RFID Reliability

• Read success rate: 98% within 4cm range
• Read time: Depends on angle

Documentation Quality Assessment

Weekly documentation provided:

• Week 2: Enclosure design process
• Week 5: 3D printing enclosure
• Week 3: PCB fabrication techniques
• Week 10: Molding & casting workflow
• Week 12: Networking implementation
• Week 13: Web interface development
• Week 14: CNC stone machining

Design Implications

Tangible AI Interfaces

This project demonstrates that physical embodiment significantly impacts AI interaction quality:

• Trust & Vulnerability: Natural materials created psychological safety that screen-based interfaces don't provide
• Mindful Interaction: Physical object encouraged slower, more thoughtful engagement vs rapid-fire phone interactions
• Ritual Creation: Picking up rock became a deliberate act of entering reflective space

Implication: AI wellness tools may benefit from tangible interfaces rather than pure software solutions

Collective Memory Systems

The COMMUNITY GARDEN mode revealed interesting dynamics:

• Temporal Bridging: Objects can carry conversational context across time and people
• Privacy Paradox: People willing to share deeply when mediated by physical object + AI abstraction

Implication: Physical objects with embedded AI memory could facilitate community reflection in ways digital platforms cannot

Technical Implications

Embedded AI Feasibility

This project proves that sophisticated AI interactions are possible on low-cost embedded hardware ($45/unit):

• ESP32-S3 sufficient for real-time audio + API orchestration
• Cloud AI APIs (Gemini, OpenAI) provide enterprise-quality models without local compute

Implication: Smart object design can leverage cloud AI without requiring smartphone apps as intermediaries

Open Source Reproducibility

All fabrication methods used in this project are accessible via fab labs worldwide:

• CNC PCB milling (no cleanroom needed)
• Desktop 3D printing
• Silicone molding (low material cost)
• Tabletop CNC for stone work

Implication: This design pattern is globally replicable using distributed fabrication infrastructure

AI as Facilitator vs Replacement

Rock Talk positions AI as conversation facilitator rather than companion:

• Generates questions that connect people to each other
• Doesn't pretend to be sentient or human-like
• Explicit about its role as tool for reflection

Implication: This "AI as tool" framing may be healthier than "AI as friend" models in emotional wellness applications

Therapeutic Applications

Potential use cases beyond this prototype:

• Grief Support Groups: Passing rocks to share memories of loved ones
• Corporate Team Building: Facilitated vulnerability in professional settings
• Educational Settings: Peer reflection in classrooms
• Therapy Practices: Homework tool between sessions

Implication: Tangible AI objects could augment (not replace) human-led therapeutic practices

Download Complete Project Files

• ESP32 Firmware (Arduino)
• KiCad PCB Design Files including images
• Fusion 360 3D Models
• Node.js Web Interface
• Component Test Code

Thank You

• Neil Gershenfeld: This was the BEST class I every had. Truely powerful class that will change my next life decision.
• CBA Staff: EVERY training, troubleshooting, and all the moral supports. I want to know how Professor Neil can gather all the amazing and nice people together in this place.
• HTMAA Classmates: Inspiration, commiseration, and late-night lab companionship🌝 Especially CBA shop cohort was super supportive to each other, and you guys were my another TAs. Thanks for answering all kinds of qeustions.