Deep Intermodal Video Analytics

Intelligence Value

The DIVA program is creating automatic activity detectors that can watch hours of video and highlight the few seconds when a person or vehicle does a specific activity (e.g., carry something heavy, load it into a vehicle, then drive away). These activity detectors work in single-and multi-camera streaming video environments, and can be used to improve video forensic analysis and real-time alerting of threat scenarios (e.g., terrorist attacks and criminal activity)


Monitoring security video at airports, border crossings, or government facilities is critical for security and public safety, but camera network operators are overwhelmed with the volume of video to monitor in real time, or forensically after incidents have occurred, to identify relevant activities.

While automated object detection in imagery is now possible at levels that rival human performance, these capabilities have not extended to activity detection. The DIVA program is developing Artificial Intelligence (AI) solutions for activity detection using multi-dimensional spatio-temporal processing, graph networks, behavior prediction, and transfer learning. Approaches for simulating and collecting big data are being researched to provide the data needed for AI application. 

DIVA is a four-year program that began in September 2017 to develop activity detection capabilities that will have a significant impact on security video monitoring workflow by providing automated alerts, enabling a single operator to monitor a multi-camera network for user-defined threat activities or other activities of interest. DIVA will facilitate forensic analysis as well. To date, DIVA has built detectors for specific defined activities that surpass human detection capabilities, detecting more than twice as many activity instances as were found by a human control group consisting of crowdsourced labor. The ultimate objective is to detect 75 percent of activities while raising a false alarm only two percent of the time.

Related Publications

To access DIVA program-related publications, please visit Google Scholars


Contact Information

Program Manager

Main Office

Related Program(s)

Broad Agency Announcement (BAA)

Link(s) to BAA


Solicitation Status


Proposers' Day Date

July 12, 2016

BAA Release Date

September 13, 2016

BAA Question Period

September 13, 2016 — October 12, 2016

Proposal Due Date

November 7, 2016

Program Summary

Testing and Evaluation Partners

  • National Institute of Standards and Technology
  • Kitware Inc.

Prime Performers

  • IBM
  • Carnegie Mellon University
  • University of Maryland
  • Johns Hopkins University
  • Systems and Technology Research