An Human-AI Spatial Playground with Linguistic Blocks in VR for Easing Rumination

This week, I tried to bulit a VR platform leveaging generative AI to support mental health.

Definition of space-aware interface design

1) make users’ writing can be manipulated (push away/ turn over) in the forms of sentence/word blocks
2) the interface create a free and comfortable atmosphere for users to look at their writings
3) the 3D floating questions and blocks make the users feel that they are able to break out of their negative thinking patterns.

Feature 1: Floating Questions

To provide vivid perspectives of writing down a life event, instead of the traditional turns-taking dialogs, multiple questions will pop up and float around the globe to encourage people to write more without limiting them in one specific dialog flow.
the user to type a sentence, and the system will generate a GPT-3 prompt for the keyword of the sentence, which will automatically generate some questions. To ensure that the questions don't trigger negative emotions, but rather spontaneous expressions, we checked to see if the questions were emotionally neutral before popping up.

Feature 2: Zoom-out Rumination Blocks

To get users to think outside of the author mindest and re-examine negative words detected by our model in their own writing, we packaged these words into individual blocks and let the squares float away. By changing the distance, this feature will guide the users to review their writing as a bystander.

Feature 3: Collision and Remixing

Based on all these negative words, our system will suggest some corresponding positive words for text style transfer, with the goal of reversing the sentiment polarity of the sentence. If the negative blocks are turned over, the opposite side with words for positive reframing are shown in the surface of the floating blocks. And users can take the authority to remix the negative sentence and positive words. By clicking on the blocks, the users can reframe rumination into a positive version

Overall Implementation

The interactive prototype is implemented in Unity, and all text processing functions involved are implemented by OpenAI Text completion API with custom text prompts.

Question Generation for Feature 1

The question generation scene consists of two components: User input interface and question display. Users will be prompt to type their thoughts in the input field and the input interface translates the sentences onto floating cuboids to fit the spatial design. Meanwhile, question display projects prompting questions onto the surface of the constantly rotating spherical structure, so that users are encouraged to write more by browsing a number of questions instead of being cued to directly answer the questions.

Positive Reframing for Feature 3

In positive reframing scene, camera is repositioned to reveal the chair to provide users with a psychologically comfortable feeling while viewing the sentence they choose. Users will be prompted to choose a positive word, and the program will reframe the sentence with attempt to relate it to the chosen word.

Email

yzihan@media.mit.edu