The US Army's new mixed reality heads-up display can translate languages and see through smoke - turning every soldier into Iron Man

US Army

IVAS Soldier Touch Point II at Fort Pickett, VA.

• Business Insider recently participated in a soldier touch point event, an exercise intended to inform the development of new technologies, at Fort Pickett in Virginia.
• During the technology demonstration and soldier-centered evaluation, we were given the opportunity to see firsthand what the Integrated Visual Augmentation System (IVAS) heads-up display can do.
• Using the HUD built by Microsoft, we planned a mission, took photos of sensitive information found on the battlefield, identified and registered detained enemy combatants, translated signs, and battled our way through a dark building full of threats.

The US Army is partnering with Microsoft to develop a mixed reality heads-up display for the dismounted soldier, and Business Insider recently had the chance to put this device to the test at Fort Pickett in Virginia.

Microsoft was awarded a $480 million contract last fall to develop augmented reality headsets for ground troops. The tech giant, which beat out other competitors like Magic Leap, has been working closely with the Army to develop the Integrated Visual Augmentation System.

IVAS is a HUD based on Microsoft's HoloLens that offers soldiers immediate access to dozens of valuable combat tools in a digital space, such as mapping, orienteering, target tracking, facial recognition, and text translation.

As IVAS is still a work in progress, the Army is not quite ready to compete with the fictional Tony Stark and his Iron Man suit, but, as BI saw firsthand, the base software is already capable of performing complex tasks that soldiers say would benefit them today.

At the recent soldier touch point event, a soldier-centered evaluation intended to inform development of new technology, our "squad" used the headsets to plan a mission using 3D augmented reality maps, take photos of sensitive information on the battlefield, identify (with confidence readings) and register detained enemy combatants, and translate Russian signs.

As we stormed a building at our objective, we were also able to conveniently use all of the capabilities of the Enhanced Night Vision Goggles - Binocular, the most advanced night vision goggles in the world today, using Microsoft's see-through waveguide display technology together with the digital low light and thermal sensors developed by various other companies.

At the heart of all night-vision goggles is an analog image intensifier tube that has to rest in front of the user's eyes. When they are turned off, the user must move the goggles out of the way because it is impossible to see through them.

With IVAS, the sensors are digital and positioned along the brow line, creating a low-profile display with a wide field of view (roughly 80 degrees instead of 40) users can see through day or night with low-light or thermal sensing capabilities available as needed.

IVAS, like the ENVG-B, also connects to the Family of Weapon Sights - Individual, which moves the reticle into the display for rapid target acquisition, improved marksmanship, and effective engagement with reduced exposure to enemy fire.

During the training operation, we were able to use the FWS-I and the HUD thermal imaging capabilities to put fire on a threat holding a civilian hostage in a dark side room without putting ourselves or our "squad mates" in danger.

After clearing all targets inside, we exited the building under the cover of our own smoke, the HUD's digital sensors allowing us to see through the obscurant without any problems.

The assortment of combat capabilities available to us during the exercise were supported by on-body computing devices, as well as mobile network units, and powered by a conformable wearable battery offering about 6-8 hours of power.

The Army, together with Microsoft, tested IVAS Capability Set 1 earlier this year. The IVAS headset we worked with during the most recent soldier touch point event was Capability Set 2. The Army plans to test Capability Set 3, the first all-weather, ruggedized, form-fitting IVAS HUD, next summer, with the Capability Set 4 testing expected at a later date.

The Army anticipates fielding IVAS in 2021, although the possibility of an extended continuing resolution could delay the ongoing work on IVAS, pushing this system's delivery to the warfighter back several months, if not years.