Undersea started from a simple demo where a single fish interacted with the user’s hand as it lead them towards a human-sized spherical aquarium featuring a coral reef and many varieties of fish. Though that initial demo was fairly simple, it resonated with the team as an opportunity to showcase the potential of spatial computing. It was built using Unreal Engine 4 and Vulkan 3.1 mobile on Magic Leap One.

I was brought into the project to help design an experience that went beyond what had been achieved with the original demo, additionally I lead all Interaction Design and UI work by sharing learnings and best practices as well as producing user flows, menu wireframes and final assets.

This project was part of an ongoing series of curated experiences that allowed internal developers to advance their understanding of creative, technical, and production challenges while delivering magical user experiences and sharing those insights with the broader creator community.

We wanted to provide users with an opportunity to relax, as they observed their room transform into a beautiful photo-realistic coral reef biome. We utilized wall surfaces to generate different “vistas”, which would serve as windows into an underwater world and as the gateway for AI-driven creatures to freely navigate in and out of the user’s space.

Early on we observed that the realistic appearance of the fish and coral increased user expectations, who couldn’t resist the urge of interacting with the fish. As such, we made the determination of not utilizing the controller after users had made their coral reef selection, allowing users to interact with the virtual content using their hands.

Fish were given relatively simple AI pathing rules which considered the user’s position, their hands and their physical space. Fish eyes featured a “look-at” system, which enabled them to make eye contact with the users as they navigated around them, this made fish feel more alive and believable. Different fish were designed to behave distinctly from one another, some fish would flee as users approached them, others would graciously navigate towards the user’s hands as they nibbled their fingers.

The most complex fish were dubbed “hero creatures” these featured unique flourishing moments to reward user observation. For example, a blowfish would gently navigate into a room and after specific conditions were met, it would start inflating its body. These kind of behaviors are truly magical if one ever is lucky enough to observe them out in the wild.

This experience became particularly important to me because it enabled people who had never been underwater to get a glimpse of the beauty of the oceans, while establishing a more personal connection and deeper understanding of the importance of these ecosystems. It’s extremely gratifying to be able to work on an experience that transcends entertainment and provides educational value to the audience!

Undersea has been selected as a 2020 SXSW finalist in the Innovation Award Finalist.


Sr Designer