A creative engine that allows writers
to visualize and explore the worlds in their stories
through Mixed Reality
to visualize and explore the worlds in their stories
through Mixed Reality
Microsoft Research | 2017 Design Expo
Muse is a storytelling engine that helps writers find inspiration by projecting 3D virtual scenes onto their environment. It allows writers to visualize, test, and explore the worlds in their stories. While interacting with virtual holograms together with the physical space, users can alter the storyline by tapping the physical orbs or through voice commands.
From our initial exploration, our team keeps refining our research questions: What is the relationship between storytelling and visualization? How can we help people engage with this relationship in interesting and evocative ways?
We look deeper into a framework of storytelling, which includes five key elements: Character, Setting, Genre, Plot and Conflict. Based on this framework, we want to provide writers the freedom to build the story creatively by entering the Character, Setting, Genre as anchors. On the other hand, our storytelling AI engine will generate new plots and Conflicts to inspire writers.
We started ideation phase by keeping ask "WHAT IF" questions: "what if visual stories could be captured through physical objects?", "How could this interact with voice narration and personal memory?", "How can these visualizations respond to touch and motion."
We build multiple narrative and scenarios to communicate and share our ideas within the group. After getting enough wild concepts, we re-evaluate each concept by defining the detailed interaction moments in mixed reality world.
We explored many key interactions through the process of creating the narrative and storyboard. Because we were building a story for a creative storytelling engine, we explored several new ways to tell the scenario. We animated the storyboard and add voice over to convey the concepts in a more intuitive way. We also found this way useful to reveal more interaction details and visualize them in different settings.
Muse collaging together different elements of a story into an interactive holographic world and integrating them into your environment. There are three major steps of exploration with MUSE. Writers began by identifying anchors — character, setting, and genre — which are captured by three real, tangible orbs. After deciding on your anchors, writers can then get up and place them around a physical space , allowing you to control over the magnitude and legibility of your virtual world. Moreover, writers can further shape and interact with the world by verbally giving commands or pause at certain moment by scaling it down .
Video prototype is one of the most intuitive method to visualize and showcase design in 3D hologram space. Our team created a concept video as the key artifacts to communicate and showcase our concept. We tried to keep the magical and wizardry parts of design in Mixed reality in the video by overlapping more animation effects and 3D-modeled visual elements into the real-world footage.
