Key Considerations for Algorithm Training and Real-World Simulation

Computer vision (CV) and algorithm training is a major challenge for the Department of Defense (DoD) as they look to make America a leader in military artificial intelligence (AI) applications. Fueled by the National Defense Authorization Act and executive orders, the DoD has increased funding research to develop this emerging technology. In order to train algorithms effectively, engineers need to create never-before-seen models and simulations (M&S) that accurately represent real-world environments (RWE). A lack of prior work, research, and real-world data often force engineers to take a “learn-as-you-go” approach to development, while COVID-19 has limited the ability to whiteboard ideas by turning all interactions virtual.

Working with DoD and partner organizations, ECS recently developed AI/CV algorithms to help autonomously control the flight and sensor capabilities of an unmanned aerial vehicle (UAV). Through the creation of a multidimensional, physics-based, simulation model, our experts have identified some best practices to ease the burden of AI/CV development.

Here are five key development practices to consider when implementing AI/CV algorithms:

Open System Architecture (OSA)

Incorporating OSA is a must for any AI/CV simulation that aims to model changing real-world physics. The architecture helps create a highly cohesive simulation environment that is flexible, scalable, and easily modified due to its modular nature. Without OSA, whenever environmental factors need to be changed, the simulation must be redesigned and rebuilt. But with the “plug-and-play” capabilities of OSA, modifying or reconfiguring a simulation is more efficient, resulting in a long-term reduction of operational costs.

Edge Computing and Mobility

As the needs of a mission evolve, AI/CV solutions need to be flexible, scalable, and mobile, but systems that require hardware buildouts can be cumbersome to expand. They often result in increased project costs and delayed scheduling timelines. Edge computing, however, enables engineers to better adapt the training and simulation environments to DoD’s operational needs. By bringing computation close to both data resources and the user, edge computing creates a mobile, flexible solution that offers:

With edge computing, organizations need not invest in costly hardware to run training and simulation environments. Instead, software is run on existing server networks or even in secure GovCloud. This flexibility enables simulations to run in different classification environments without the need to create a separate hardware buildout for each. Engineers can manage controlled unclassified information (CUI), secret, and top secret/sensitive compartmentalized information (TS/SCI) efficiently and securely all within the same network. In one case study, we found that utilizing a mobile environment resulted in a 20 percent reduction in the cost of procurement.

Computational Power: Central Processing Unit (CPU) vs. Graphics Processing Unit (GPU)

Running physics-based simulations is computationally intensive. While CPUs can handle a wide range of tasks quickly, they are limited in the number of tasks they can run concurrently. A GPU, however, is designed to quickly render high-resolution images and video at the same time.

In many ways, GPUs are faster and more efficient than traditional CPUs. They are better at collision detection, data preprocessing, and N-body calculations which predict the motion of individual bodies within a group of objects. For the physics-based simulations required by AI/CV models, GPUs are the better design choice for performing these critical functions.

Verification Validation & Accreditation (VV&A)

The goal of any simulation is to mimic or predict real-world behavior, but once a model is built, engineers need a way to test and measure its accuracy. VV&A is the method by which we evaluate those key metrics. Without effective VV&A, engineers cannot ensure a model accurately depicts a real-world environment, which nullifies any data the simulation uncovers. Many engineers falter in this crucial testing process.

VV&A has its challenges, especially with regards to validation, possibly the most important aspect of the VV&A triad. Since engineers are often evaluating new capabilities and technologies, they need to be able to determine if model accurately simulates a RWE even when no historical data exists. At ECS, we follow a framework put forth by Bob Eberth, former M&S advisor to the Marine Corps Warfighting Lab. In his words, “validation is the failure to invalidate.” If an engineer can uncover logic flaws that invalidate the model, the model is incomplete. The only way for an engineer to prove an AI/CV model is valid is through the effective testing and evaluation (T&E) of alternative hypotheses.

A New Frontier of DoD Applications

Engineers have only begun to explore the possibilities of AI/CV within DoD, Homeland Security, and the government. The future of AI/CV will transform national security from supply chain management to autonomous vehicles and ships. As the technology matures and expands, ECS continues to lead its development. Using these “five key development practices,” ECS is creating new and advanced AI/CV solutions to power the next-generation of autonomous robotics and intelligent systems.