top of page
Vive
AR experiences with Video Passthrough
Role: Concept Vision, Research, IA Flows, UX, UI, Storyboards, 3D Assets for build, Tech Demo
Responsibility: Research, IA flows, Concept Designs, 3D Asset creation, UX presentations, Oversee development of Demo in Unity
Process: Research, 3D, Iterate Solutions, Create Builds, Support Developers
Solution/Result: Proof of concept for Vive "Lego Experience"
This was a conceptual AR Experience. The project to create models in realtime with AR overlay reference images. This would also lay the needed groundwork for the development of a more robust "building experience" in AR/VR. If successful, these interactions would also apply to any "modular" building experience in AR. From building molecules with Virtual and Real models, to installing and repairing engine components in vehicles. The main idea was to enable the system to recognize parts as they are brought into view, then provide correct visual instructions on where the items should be placed.
Storyboards
For the next phase, I created storyboards of several interactions that we could choose to develop and research. One of the experience was a way to use hand gestures to build with lego bricks in VR (see below)
Starting the Demo: Teaching the HMD to see
For phase one, I thought we would start with larger "Duplo" bricks and build a demo for assembling a model with AR instructions. I created 3D models of individual "Duplo" bricks of various sizes and styles to provide a good set of reference models for the experience. I also created step by step recreations of each stage of the model. We needed to have a reference model of each stage of the build for the HMD to recognize the object, and pull the AR representation from the data base and display it.
This video shows the first step on getting the HMD to superimpose our 3D models of the objects over the video passthrough images
Anchor 1
This video shows the HMD rendering a 3D model next to it's video passthrough example and scaling the size correctly.
Starting the Demo: Steps for building the model
For our first demo, we created a simple step by step build. The HMD would display the required piece in Situ, and the user would mimic the piece used and positioning as the model was created.
In the final part of the demo, we superimpose the AR components directly over the real parts of the model in Video Pass through. This would be the first step in getting overly instruction graphics along with the video passthrough.
Results
The Vive Brick experience showed that we can track real lego pieces and successfully superimpose a virtual model to act as a reference in building the Lego model.
Reaction
4k passthrough was good enough to allow us to proceed with a more refined demo experience.
bottom of page