Created a virtual reality application for public speaking practice
3D Product Design / UX Research
1% of the total population who suffer from stuttering often find social scenarios stressful.
Due to the lack of immersion, current treatments cannot prepare stutters effectively for real-world scenarios. Along with four other team members, I created VRbal with a mission to make practicing for these scenarios easier with artificial intelligence in virtual reality.
Current treatments often include practicing “pretend-play” (Applied Behavioral Analysis) with therapists to prepare them for real-world scenarios. However, these techniques do not awaken similar emotions in the the patient as would the real situation due to the fact that it is not considered a sufficiently realistic simulation of the scenario.
Primary Research: We interviewed with Speech-Language Pathologists (SLPs) and stutterers to understand the current treatment and what virtual reality can provide.
“Stuttering is like an iceberg. The only way you’ll ever get over your fear of stuttering is by addressing what’s underneath the surface level.” (Mark, Speech-Language Pathologists)
"It’s difficult to apply what I learn [from the therapy], because the environment [in the therapist's office] is quiet and I was with someone I trust. But when I go outside, there are loud noises and I meet new people." (Daniel M., user)
"When I get used to dealing with an actual situation, my stuttering can be milder." (Daniel M., user)
Systematic Desensitization: is a type of treatment that involves removing the fear response and substituting a relaxation response gradually using counter conditioning. This is the basic of VRbal's training method.
Competitor Analysis: Mimerse, CleVR, Virtually Better and SimSenSei are some of our competitors on the market. However, most of these tools either require a therapist in presence or lack customization.
Before designing a high fidelity application using Unity3D and IBM Watson, we created storyboards and Google Cardboard prototypes.
Storyboard & Google Cardboard
I drafted the first 2D storyboard and a video for the Google Cardboard. The first prototype included one abstract relaxation scene and another in an auditorium where the user could practice public speaking. We learned everyone’s definition of relaxation is different. While sitting at a beach is peaceful for some, it’s not for others. As a result, we decided to let the users customize their own relaxation scene by conversing with the AI agent.
Unity3D & IBM Watson
One of my main responsibilities was designing the 3D user interface for all the levels within VRbal.
1. Intro: is where the user first entered VRbal. Here the AI agent, powered by IBM Watson, asked the users about their preferences for the relaxation scene that follows.
Design Decision: Why the space theme?
In VRbal, users switch back and forth between the two worlds of the relaxation scenes & public speaking scenes. The intro is therefore designed to be a fantasy space between those two worlds.
2. Relaxation Scenes. In the current demo, we have two options, forest and beach scenes, each comes with two activities: deep breathing or vowel speaking exercises to help users lower their stress level before practice public speaking.
Design Decision: Why using low-poly instead of photo-realistic aesthetics? (Ask me in person!)
3. Public Speaking Scene. Using 360 cameras, I created a few different levels for this scene, ranging for easy (with no audience and background noise) to difficult (more audience and a lot of background noises).
We conducted user tests with 20+ stutterers from the National Stuttering Association in New York City.
After each round of prototyping, we interviewed potential users to identify the pain points. We carefully go analyzed every feedback we received before incorporating them into VRbal. We learned the different attributes of a public speaking environment that can induce stress, including the number of people, background noise level, and interrupted events.
VRbal is an AI-powered educational tool that allows people with communication disorder practice public speaking in VR.
Artificial intelligence enables a virtual environment that adapts to the user’s needs as training progresses. Unlike weekly therapies, AI tracks user feedback and her focus of attention so that they can learn in a customized way whenever they are ready to train. These technologies provide a socially safe and optimized learning experience that can build communication skills.
Although not meant to replace therapy entirely, VRbal aims to replace Applied Behavioral Analysis practice. Users can user VRbal to practice anytime and anywhere they want.