The New Reality: How augmented, virtual and mixed reality will enhance the future soldier

2016 is set to be a significant year in the development of virtual reality (VR) technology, where major industry players such as Oculus, Sony and HTC are launching or have launched key consumer VR products.

So, what are virtual reality, augmented reality (AR) and mixed reality (MR)? How will they change the way we experience the world, and how will they shape future land warfare?

The reality continuum spans from simple overlaying of information into the users vision, to full immersive 3D virtual reality at the other. In between is MR, where the real and virtual worlds are merged to produce new environments, where physical and digital objects co-exist and interact in real-time, with projections that have been described as looking as real as the objects in the room surrounding them.

AR/VR have been projected to be a $150 billion market as early as 2020. Almost all the big tech companies, as well as thousands of new start-ups, have AR/VR projects in the pipeline. AR/VR is going to a game-changing technology for consumers and industry, but what does it mean to the future military?

The way that this technology is used will be unpredictable. Who would have thought in 2007, when the first iPhone hit the market, that by 2016 iPhones would be used as powerful IO, recruiting and encrypted communication tools; or iPads would be used to aim mortars, and provide for satellite imagery and mapping? ADF commanders have started using the latest iPhone business apps to better communicate and interact with their soldiers.

Applied Research Associates has been collaborating with the US Defence Advanced Research Projects Agency (DARPA) for the past seven years to create an AR system named ARC4, which was highlighted in a previous land power forum article. But the uses for AR are not limited to the warfighter—the technology is expanding across an array of industries, providing real-time heads-up information to drivers, warehouse operators and maintenance technicians.

The military has a vast fleet of diverse vehicles and equipment. AR will allow a technician to work on a wider array of vehicles, with less training. AR has the ability to identify parts via object recognition, and then overlay both instructions and visualisations into the users view, while keeping both their hands on the job. For more complex repairs, the technician can video link to a remote specialist, who can speak to them and use pointers and markups to guide their work.

The same concept applies to treating medical casualties in the case where a trained medic is not available. The quality of advice that can be provided through AR is far greater than can be conveyed by voice alone.

The advantage of AR to a storeman is much greater when operating in an unfamiliar environment, like when conducting 6-month overseas deployments—where there is less time to become familiar with the layout of a warehouse.

VR also has an important place in the future military, especially when it comes to training and simulation. VR brings another dimension beyond what can be provided by the current weapon training simulation system (WTSS). While WTSS is a valuable tool for marksmanship practice, VR allows for an immersive training experience, akin to what is provided in a flight simulator. VR uses a technique call redirected walking, where subtle alterations to turn angles allow the user to believe they are operating in a vast environment, when really they are inside a 20 x 20 m room. Other tricks create a multitude of diverse spaces, out of a limited number of simple objects. This allows the system to train and test soldiers across a wide variety of scenarios.

AR, VR & MR are set to be world-changing technologies. The ADF should leverage off industry to help deliver AR for logistical applications, while continuing to explore and develop the technology to aid the future soldier.