FOCUS

Forecasting Counterfactuals in Uncontrolled Settings

Intelligence Value

FOCUS enhanced analysis tradecraft by improving analysts’ ability to learn lessons from the past. FOCUS helped analysts correctly answer the question, “how would history have been different if actor X had instead done action Y on date Z?”

Summary

Counterfactual forecasts -- statements about what would have happened if different circumstances had occurred -- are the foundation for lessons about what to do in the future. For example, analysts performing a postmortem review of an intelligence failure might conclude the Intelligence Community could have avoided the failure if it had employed different methods based on prior experience and best practices. Unfortunately, research across multiple disciplines has concluded that experience-based best practices are generally ineffective. There has been no systematic prior research on improving counterfactual reasoning or lessons learned. The FOCUS program aimed to develop practical, accurate, evidence-based techniques for counterfactual forecasting.

Because we can’t re-run history in the real world, FOCUS methods were evaluated by having hundreds of people make predictions about outcomes in complex simulations.

Proposers' Day Briefings

Related Publications

To access FOCUS program-related publications, please visit Google Scholar.

FOCUS Logo

Contact Information

Program Manager

Main Office

Broad Agency Announcement (BAA)

Link(s) to BAA

IARPA-BAA-17-08

Solicitation Status

CLOSED

Proposers' Day Date

October 19, 2017

BAA Release Date

March 29, 2018

BAA Question Period

March 29, 2018 — May 1, 2018

Proposal Due Date

May 30, 2018

Program Summary

Testing and Evaluation Partners

  • Johns Hopkins University Applied Physics Laboratory

Prime Performers

  • Charles River Analytics
  • Raytheon BBN Technologies
  • The University of Pennsylvania