Immersive Synthetic Memories (ISM)
Virtual Reality Installation
Memory is a fragile technology of the mind, distorting and fragmenting over time. Immersive Synthetic Memories (ISM) utilizes Generative AI to reconstruct lost or undocumented moments through immersive video and images. This research project is a collaboration between USC Mobile & Environmental Media Lab and Domestic Data Streamers (DDS) on their Synthetic Memories Project.
Exhibited: 2024
Platforms: Meta Quest 3
Tools: Generative Image Systems, Gaussian Splatting
Role:
Experience Research, Pipeline Design, Prompt Engineer
Result: Premiered at Flux Festival, published in Leonardo Journal (MIT Press), and featured in the Berggruen Institute’s Vaster Than Empires III retreat.
01 | Context
This project was a collaboration with Domestic Data Streamers on their Synthetic Memories project, which reconstructs lost or fragmented memories through Generative AI. While their existing pipeline produced 2D representations of memories based on interview data, our mission was to make these moments immersive, allowing participants to step inside their memories in 3D space with audio.
Our research focused on two key goals:
- Understand how immersive environments affect emotional and cognitive recall.
- Design a streamlined, efficient pipeline that could generate these immersive experiences in real time.
02 | Final Pipeline
The process begins with a one-on-one interview, where participants describe a vivid memory in detail. Using AI-generated visual prompts, these recollections are iteratively refined until they align closely with the participant’s perception.
Through experimentation with different ways to spatialize memory, we developed a pipeline that leverages 3D stereoscopic 180° video and spatialized audio to evoke a strong sense of presence and emotional resonance. This immersive format allowed interviewees to not just recall, but relive their memories in a comforting embodied way.
The experience integrates two complementary visualization methods:
180° VR rendering
Anchors participants within a richly spatial environment
Gaussian splatting
Generate fluid, volumetric reconstructions of memory spaces
03 | Results
To evaluate both emotional impact and technical feasibility, we tested the immersive experience with participants during the installation. Each person received a 2D Polaroid reconstruction of their memory before entering the headset. In nearly all cases, the immersive version, whether rendered as stereoscopic video or through Gaussian splatting, evoked a noticeably stronger emotional response, reinforcing the potential of spatial immersion in deepening memory recall.
Although speed was not the primary focus, we successfully generated 15 immersive memories within just a few hours. Compared to DDS’ original pipeline, which typically produces 3–4 memories per day, this streamlined approach demonstrates how immersion can be delivered efficiently without compromising emotional resonance.
DDS is now exploring ways to incorporate this pipeline into future iterations of Synthetic Memories, allowing their team to continue focusing on restoring lost memories while seamlessly integrating real-time immersive experiences.