Virtual Reality ISS On-board Training
On-board training (OBT) is a valuable training tool for the crew members on-board the International Space Station (ISS). OBTs are designed to complement ground training and provide rapid assistance in unforeseen complex circumstances or refresh knowledge and skills before execution. Virtual reality lends itself to this as a suitable medium for training complex tasks effectively and enables delivery of so called ‘just-in-time’ training as it shortens training time on the ground, which is a decisive advantage in the field of space exploration.
For one of these on-board trainings, a virtual reality application was developed for the 2021-22 Cosmic Kiss mission to prepare German ESA astronaut Dr. Matthias Maurer for an upcoming remove and replace procedure. The goal was to test whether VR is a suitable medium for this type of training for astronauts aboard the International Space Station.
A standalone VR headset, such as the Oculus Quest used for this training application, relies on many sensors to accurately sense the environment and calculate the user’s position and orientation within it. Embedded sensors such as accelerometers and magnetometers in IMUs, which are needed for such tracking, do not work properly in microgravity. This leads to problems and errors in displaying the intended training content, including unwanted drift, jitter, and flicker. For this reason, a solution needed to be developed to minimize these effects.
Ultimately, the VR headset performed better than expected, although it was clear that more work is needed to make VR a seamless training tool in orbit.
https://www.esa.int/ESA_Multimedia/Images/2022/05/Virtually_reality_in_orbit
My contribution
- Principal Payload Developer for parabolic flight prototypes and training application
- Integration of an IMU anti-drift solution
- CAD retopology and re-texturing for Oculus Quest target device
- Procedure conversion from PDF to Immersive VR content
- User Interface and Interaction Design
