La Fondation Dassault Systèmes / MBARI 2020-2021

Virtual Reality Control Room for Underwater Remotely Operated Vehicles

Monterey Bay Aquarium Research Institute (MBARI) uses Underwater Remotely Operated Vehicles (ROVs) to conduct scientific dives to research the deep ocean at depths up to 4000m, improving our understanding of the deep ocean and finding ways to mitigate our impact on it. The 2020-2021 a Fondation Dassault Systèmes / MBARI SCOPE Team developed a real-time virtual reality (VR) control room that has the potential to streamline both ROV piloting and scientist-pilot collaboration. Typical ROV control rooms consist of a wall of fixed monitors, each displaying a separate piece of telemetry data. MBARI’s ROVs have been equipped with a 180° 4K stereo-pair fisheye camera to aid in scientific study and generate educational content. Our project, an early prototype, uses live footage from this camera to compute the re-projections necessary to create an immersive, 1:1 scale, 3D view of the ROV’s surroundings in VR. Providing ROV pilots with a wide field of view, without the distortion that results from projecting raw footage onto a flat screen, should improve pilots’ spatial awareness during operation. To begin incorporating the functionality of existing control rooms into our application, we project real-time telemetry data into the 3D environment and provide multi-user support to allow pilots and scientists to collaborate in VR. Our app also includes hand and gaze-based controls that enable pilots to rearrange the displays, a feature not possible with fixed monitors. Since pilots have individual preferences for how these displays are arranged, we included an option to save their display preferences, allowing for seamless transitions between pilots. We demonstrated our application during the operation of MBARI’s MiniROV in a 1 million liter saltwater test tank facility, and we anticipate testing our application at sea later in 2021.  

Faculty Advisor

Lynn Andrea Stein

Team Members

Everardo Gonzalez

Erika Lu

Amy Phung

Nathan Shuster

Cameron Wierzbanowski