(Controlled) Fortune Teller

Fortune-telling device where Arduino touch sensors and an OLED display connect with Gemini that listens to your touch, whispers today’s destiny, and twists it into optimism, wisdom, mean, or mystery — however you like.

Embedded Programming

Course Source

Flow

The project combines hardware (touch sensors + OLED display) with a Python server that connects to Google’s Gemini LLM. The interaction flow is:

User presses the start pad → Arduino shows today’s date and requests an initial fortune.
Python requests Gemini for a short daily fortune and sends it back.
Arduino displays the fortune on its OLED screen.
Arduino sends the tone request + current fortune to Python.
Python rewrites the same fortune in the requested tone and sends it back to Arduino.
This creates an interactive loop where the fortune evolves with the user’s choices.

Fortunes

Original::A small act of kindness today will have a ripple effect. Embrace the opportunity to brighten someone's day.
Wise::A whisper of generosity, a feather-light touch of grace offered into the void this day...its echo will resonate in unseen currents. Embrace the chance to become a glimmer in the gathering shadows.
Mean::Alright, listen up, buttercup. That pathetic little act of "generosity" you're planning? It's a drop in the bucket, a pathetic attempt to look good.
Optimistic::Hey there, sunshine! That kind gesture you're thinking about? That's a wonderful seed you're planting! While it might feel small now, it has the potential to blossom into something
Mysterious::Ah, dear one, I sense a gentle benevolence within you. That impulse toward kindness, that selfless act you contemplate... nurture it. For it is a seed, small perhaps in its present form,

Understanding / Making the Electronics

Electronics

HUGE THANKS TO CBA Section AMAZING TA ✨Quentin✨

The design of the electronics board is here

The arduino board setting is here

Board: QPAD21
Touch pads: Used for detecting user input (start + 4 tones)
OLED Display (SSD1306, 128x64): To show fortunes and status messages
LED: Simple indicator when touches are active

I putted the LED in the wrong direction and had to detatch them and do again. Also, since USB connection is really fragile, needed to re-solder it.

Programming

You can find the source code from here

Arduino IDE Programming

Touch Detection: Using the Adafruit_FreeTouch library, raw touch sensor values are read and converted into “pressed / not pressed” states.
When an event happens (start pressed, tone requested), Arduino sends structured messages over Serial to ask Python server to call the Gemini request.
Then, listens for the responses from the Python, and show on the OLED display.

Python Programming

The Python script acted as a bridge between Arduino and Gemini. Since boards cannot directly call Gemini (they lack HTTPS + OAuth support), I offloaded LLM requests to a Python server, using Serial as a bridge.

Serial Communication (via pyserial)

ser = serial.Serial('/dev/cu.usbmodem2101', 9600, timeout=1)
while True:
    if ser.in_waiting > 0:
        line = ser.readline().decode(errors="ignore").strip()
        if not line:
            continue
        print("👉 Arduino says:", line)

Opens the Arduino USB port (COM3 on Windows, /dev/ttyUSB0 or /dev/cu.usbmodemXXXX on Mac/Linux).
Reads messages from Arduino and writes back responses.

Gemini API Calls (via google-generativeai)

elif line.startswith("request:tone:"):
  parts = line.split("::", 1)
  header = parts[0]
  fortune_text = parts[1] if len(parts) > 1 else ""
  tone = header.replace("request:tone:", "").strip()

  prompt = f"Rewrite this fortune in a {tone} tone, keeping meaning the same: {fortune_text}"
  response = model.generate_content(prompt, generation_config={"max_output_tokens": 40})
  fortune = response.text.strip().replace("\n", " ")
  ser.write((f"fortune::{fortune}\n").encode())

When it receives request:initial: -> Send today’s date back.
Generate a one-sentence fortune using Gemini.
When it receives request:tone:...::fortune: -> Rewrite the fortune in the requested tone (optimistic, wise, humorous, mysterious).

Environment Setting

Used a .env file with python-dotenv to keep the API key secure.
Virtual environment (venv) was set up to install google-generativeai and pyserial.

Gemini Prompt

Gemini often produced long texts, so adjusted prompt and added the setting.

prompt = f"Rewrite this fortune in a {tone} tone, keeping meaning the same: {fortune_text}"
response = model.generate_content(prompt, generation_config={"max_output_tokens": 40})

Selected the wrong microcontroller profile during program upload, which corrupted the flash and required re-downloading the flash to the board.

Learning from Failures

Put the LED Diodes in the wrong direction!
Soldering is challenging!

Should shut down the Arduino serial monitor to run the Python server. Otherwise it shows:

raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 16] could not open port /dev/cu.usbmodem2101

I tried using PlatformIO instead of the Arduino IDE to make it easier to connect with the Gemini Python server, but unfortunately PlatformIO doesn’t support our board. Sun has documented the issue here.

← Previous Week