Making Data Analysis More AGILE

April 28, 2023

The amount of digital data we consume daily can often feel overwhelming. More data is coming at us than ever before in the form of emails, text messages, and social media feeds. With such an avalanche of information, it is often hard to distinguish between what’s important and what can be ignored.

A somewhat similar problem exists within the Intelligence Community (IC) and the Department of Defense (DoD), albeit at a scale far exceeding what the average person is exposed to and with far greater consequences. Specifically, today’s new era of explosive data growth poses serious challenges for the IC and the DoD in monitoring adversarial activities and predicting their future actions. This burgeoning data, which is increasing exponentially not only in volume but in velocity, variety, and complexity, already far outpaces the abilities of current computing systems to execute the complex analytics needed to extract meaningful insights in a timely manner.

In an effort to address this problem, last October the Intelligence Advanced Research Projects Activity (IARPA) launched the Advanced Graphic Intelligence Logical Computing Environment (AGILE) program. AGILE’s goal is to create computer architectures that can handle the IC and DoD’s large-scale data-analytic applications and other classes of data-intensive applications. The resulting AGILE technology should provide the ability to more accurately predict events based on the analysis of massive data from diverse sources and methods, not just forensic analysis after the event.

“Current computer technology simply isn’t designed to address today’s data-intensive problems,” said AGILE Program Manager, Dr. William Harrod. “In other words, there’s too much data and an inability to analyze and make sense of the data using today’s compute-centric computers—the problem that the AGILE program seeks to address.”

On a practical level, this means AGILE technology could potentially help predict a terrorist attack, rioting in a foreign capital, or other events with national security implications. However, when fully developed, how AGILE technology will be used will depend on each agency’s needs. For example, an intelligence analyst may decide to use AGILE technology to assist her in answering policy-maker questions on a foreign nation’s nuclear intentions, or the DoD may use real-time data with AGILE technology to help predict when a conflict could arise between two countries. Thus, AGILE technology will help inform, but not determine, the analyst’s response to policy-maker questions based on the data and application(s) processing the data, which may include artificial intelligence and other data analytic approaches.

“With AGILE, the intent is to provide a productive platform to agencies so they can apply it to various scenarios, whether classified or unclassified,” Dr. Harrod said. “That’s why, in thinking about how to develop AGILE, we discussed with agencies how to address their particular data analysis needs.”

To make AGILE technology a reality, IARPA has contracted with Advanced Micro Devices, Georgia Institute of Technology, Indiana University, Intel Federal, Qualcomm, and the University of Chicago—all of which have deep backgrounds in computer architecture and will conduct the actual research and development as part of this effort.

AGILE’s solutions will be tested and evaluated by teams from Lawrence Berkeley National Laboratory, Sandia National Laboratory, and Pacific Northwest National Laboratory—all of which have extensive experience in evaluating the design and performance of computer architectures. The hopes and expectations are that with the experience and expertise of these teams, the AGILE program will realize a fundamental rethinking of computer architectures that can revitalize performance growth trends in computing capabilities. This will in turn seed a new generation of high-performance computers that will directly benefit all taxpayers.

“While we can’t predict the final outcome of the research and development that our performers will pour into AGILE, I’m very optimistic about its prospects,” Dr. Harrod said. “And I’m excited to see how the IC and DoD will utilize this technology in the years to come.”

Making Data Analysis More AGILE