Re-Membering

Final Project Proposal

Re-Membering System Diagram

Background and Motivation

Human beings have always sought ways to preserve and revisit experiences that carry deep emotional weight. Across time, media such as vinyl records, cassette tapes, DVDs, and digital archives have offered ways to return to moments of music, film, and memory. My long-standing interest has been in extending this tradition into immersive media, exploring how interactive and computational tools can help us revisit timeless human experiences.

In my previous research, I have focused on 3D scanning technologies (photogrammetry, LiDAR, Gaussian splatting) and XR production. This work has informed my broader question: how can new forms of interactive media allow us not only to view, but to re-enter and re-experience spaces that matter to us?

Link to previous work

Project Concept

For my final project, I propose to build a table-top project titled Home Again. The setup will feature small physical objects in the form of cassette tapes. Each cassette will be linked to a specific virtual scene.

When a participant picks up a cassette and places it on the table, an image recognition or sensing mechanism will detect it and trigger a VR headset to load a corresponding scene. These scenes will be drawn from my collection of 3D-scanned environments, prepared both as photogrammetry meshes and as Gaussian splats.

Technical Approach

  • Physical Objects: Fabricate cassette-like artifacts that serve as triggers for virtual scenes.
  • Recognition Mechanism: Implement image recognition or object sensing to connect each cassette to a unique environment.
  • Scene Content: Use 3D scans of real spaces, prepared as photogrammetry models and Gaussian splats.
  • Integration in Unreal Engine: Build an application where cassettes placed on the table load corresponding VR scenes.
  • Independent Study Synergy: Connect this project with my ongoing independent study in Gaussian splatting, bringing technical research into an interactive format.

Expected Outcomes

  • A working table-top system where participants can β€œbrowse” cassettes and immediately enter virtual spaces through a VR headset.
  • An Unreal Engine app integrating photogrammetry models and Gaussian splats.
  • Documentation of the full pipeline: cassette fabrication, recognition mechanism, 3D capture, and VR integration.

Weekly Progress

week 1

Final Project proposal.

week 2

webpage template development, see template.

week 3

soldering for the first time

week 4

  • gaussian splat in web
  • Gaussian Splat Preview

  • gaussian file optimization
  • camera type comparison

week 5

simple magnet cassette mechanism

  • EDA, and research on hall sensor and RFID

week 6

board milling and WebXR

  • SRM-20 milling
  • WebXR experiment

week 9

Using rfid to open certain webpage

Tasks

  • Make the circuit board for the Memory Dock
    • Connect the XIAO ESP32-S3 to the RFID reader and retrieve the UID
    • Enable the XIAO ESP32-S3 to communicate with my headset over Wi-Fi
    • Write helper scripts to open different webpage URLs based on the UID
  • Create sample tokens
    • Cast NFC tags together with physical objects (e.g., leaves)
    • Or use an acrylic box to seal them
  • Design cases for the Memory Dock
  • 3D scene reconstruction (sourced from another class)
  • Schedule

    Week # Work Content
    Week 12 – networking and communications Make the circuit board for the Memory Dock
    Week 13 – interface and application programming Design cases for the Memory Dock
    Week 14 – wildcard week Create sample tokens
    Week 15 – final assignments and orders