Develop a novel method to measure the performance of credibility assessments
Registration for The CASE Challenge is now closed
View the winners of The CASE Challenge
Every day we make decisions about whether the people and information sources around us are reliable, honest, and trustworthy – the person, their actions, what they say, a news source, or the actual information being conveyed. Often, the only tool to help us make those decisions are our own judgments based on current or past experiences.
There are many technologies and techniques used to test credibility
But how do we know the results of those credibility tools are trustworthy?
A standardized method is needed to evaluate credibility assessments
Help us design that solution for government, industry and academia
For some in-person and virtual interactions there are tools to aid our judgments. These might include listening to the way someone tells a story, looking at a user badge, validating with other people - or in more formal settings, verifying biometrics or recording someone’s physiological responses, i.e. the polygraph.
Each of these examples uses a very different type of tool to augment our ability to evaluate credibility. Yet there are no standardized and rigorous tests to evaluate how comprehensive or accurate such tools really are.
IARPA's Credibility Assessment Standardized Evaluation (CASE) Challenge seeks novel
methods to measure the performance of credibility assessment techniques and technologies.
Credibility assessment refers to both the assessment of the truthfulness of specific claims and to the assessment of the reliability, honesty, and trustworthiness of a source of a particular claim, whether that be an individual, group, or a broader organization or entity.
This challenge is focused on the methods used to evaluate credibility
assessment techniques or technologies, rather than on the techniques or technologies themselves.
In this context, a method is a detailed plan or set of actions that can be easily replicated or followed.
In this challenge, we ask that your solution is a method for conducting a study, which includes background information, the objectives of the research, study design, the logistics and means for running the study, and details about what data would be collected if your solution were implemented.
Taylan Sen (Captain)
Credibility: Assessing the Assessment
Jeremy Martinez, M.D.
Strategic Cognitive and Mobility Room
Dr. Joseph R. Stainback IV
Realistic Reflection: The Purrrfect Cat – Ilona Palmer
Ground Truth: Checkmate Revisited – Charles McElroy
Creative Technology: Credibility Meta-Assessment with Neural Networks – Lex Fridman
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk, high-payoff research programs to tackle some of the most difficult challenges of the agencies and disciplines in the Intelligence Community (IC).