Crowdsourcing Evidence, Argumentation, Thinking and Evaluation (CREATE)
The CREATE program seeks to develop, and experimentally test, systems that use crowdsourcing and structured analytic techniques to improve analytic reasoning. These systems will help people better understand the evidence and assumptions that support—or conflict with—conclusions. Secondarily, they will also help users better communicate their reasoning and conclusions.
The CREATE program is envisioned as a 4.5-year effort that is intended to begin in September 2016. Phase 1 of the program lasted 23 months, Phase 2 will last 17 months and Phase 3 will last 14 months.
Join the CREATE study
We are looking for highly motivated, inquisitive, open-minded individuals to participate in CREATE. Please visit our experiment website to join in and help us improve analytic reasoning!
Performers (Prime Contractors)
George Mason University; Monash University; University of Melbourne; Syracuse University
COVID-19 Related Research
- Social and behavioral sciences
- Informal reasoning
- Computer science
- Structured analytic techniques
To access CREATE program-related publications, please visit Google Scholar.
- Mapping Our COVID-19 Recovery
- TRACE App Chosen as TechConnect Defense Innovation Awardee
- Crowdsourcing security intelligence
- Dr. Steve Rieber: Program aims to improve human reasoning
- The future of crowdsourcing: Integrating humans with machines
- IARPA launches CREATE program
- IARPA Program Seeks to Build Up Analytic Reasoning Through Crowdsourcing-Based Platforms
- IARPA to Host Proposers’ Day for Program on Crowdsourcing, Analytic Thinking
- IARPA looks to the crowd
- Leveraging the Wisdom (and ignorance) of Crowds
COVID-19 Related Research
IARPA’s Crowdsourcing Evidence, Argumentation, Thinking and Evaluation program developed tools and methods designed to improve analytic reasoning through the use of crowdsourcing and structured analytic techniques. These resources empower multi-disciplinary collaboration among analysts. A software platform developed under CREATE has been used to enable a geographically dispersed team of researchers to work together to analyze COVID-19 policy options related to social distancing.
IARPA-developed collaboration tool enables Australian researchers to build a Roadmap to Recovery during the COVID-19 outbreak.
April 18, 2020: An article in the April 18, 2020 issue of The Australian described a collaboration among roughly 100 researchers from Australia’s eight leading universities to analyze possible outcomes of policy decisions to relax COVID-19 social distancing restrictions. The university taskforce used a collaborative software platform developed under IARPA’s CREATE program. The software platform, the Smartly-assembled Wiki-style Argument Marshalling project, facilitates the aggregation of high volumes of complex information, analysis of consequences, and packaging of recommendations into a succinct report. For three weeks, the taskforce examined a range of questions ranging from whether there should be uniform approaches across Australia, rules for travel and border protection, preparations for reopening schools and businesses, special consideration for the elderly, rural residents, indigenous people, and citizens needing mental health support, and communications strategies to encourage public support and cooperation. The report, entitled the Roadmap to Recovery, was delivered to the Australian Health Minister and the Chief Medical Officer at the end of the April.