Bias Effects and Notable Generative AI Limitations


The U.S. Government is interested in safe uses of large language models (LLMs) for a wide variety of applications including the rapid summarization and contextualization of information relevant to the Intelligence Community (IC). These applications must avoid unwarranted biases and toxic outputs, preserve attribution to original sources, and be free of erroneous outputs. The U.S. Government is also interested in identifying and mitigating hazardous use of LLMs by potential adversaries.

The goal of BENGAL is to understand LLM threat modes, quantify them and to find novel methods to address threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more topic domains, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to efficiently probe LLM models to detect, characterize and mitigate biases, threats or vulnerabilities.


Proposers' Day Information

BENGAL Proposers' Day Registration Site Reference

BENGAL Teaming Form

BENGAL Draft Technical Description


Proposers' Day Briefings

BENGAL Proposers' Day Briefing

BENGAL Teaming Summary

9 HI Lightning Talk

Amazon Web Services Lightning Talk

Aptima Lightning Talk

BlueHalo Lightning Talk

CalypsoAI Capabilities Statement

Carnegie Mellon University Lightning Talk 

Cayuse Capabilities Statement

Cayuse Lightning Talk

Columbia University Irving Medical Center Lightning Talk

City University of New York Lightning Talk

George Mason University Lightning Talk

Dartmouth College Lightning Talk

DataCrunch Lab Capabilities Statement

DataCrunch Lab Lightning Talk

Eduworks Corporation Lightning Talk

Galisteo Consulting Group, Inc. Lightning Talk

Geometric Data Analytics, Inc. Lightning Talk

GoCharlie Lightning Talk

Gryphon Scientific Lightning Talk

Guidehouse Capabilities Statement

Guidehouse Lightning Talk

i2K Connect Lightning Talk

Indiana University Bloomington and Worcester Technical Institute Lightning Talk

International Computer Science Institute Lightning Talk

IQT Labs, Digital Safety Research Institute Lightning Talk 

Johns Hopkins University and University of Pennsylvania Lightning Talk

KUNGFU.AI Lightning Talk

Massachussetts Institute of Technology Computer Science & Articial Intelligence Laboratory Lightning Talk

McGill University and Columbia University Lightning Talk

Michigan State University Trustworthy AI Group Lightning Talk

Morse Corp Lightning Talk

Motalen Labs Lightning Talk

Noblis Lightning Talk

Northwestern University Lightning Talk

NovoMorpho Lightning Talk

ObjectSecurity LLC Capabilities Statement

Parallax Capabilities Statement

Penn State University and University of Mississippi Lightning Talk

Polyrific Capabilities Statement

Polyrific Lightning Talk

Purdue University Lightning Talk

Rutgers University Capabilities Statement

Trail of Bits Lightning Talk

Two Six Technologies Lightning Talk

University of California, Riverside Lightning Talk

University of California, Irvine Capabilities Statement

University of Maryland, Dept. of Computer Science Lightning Talk Capabilities Statement Lightning Talk




Contact Information

Program Manager

Dr. Timothy McKinnon

Broad Agency Announcement (BAA)

Link(s) to BAA


Solicitation Status


Proposers' Day Date

October 24, 2023

BAA Release Date

November 17, 2023

Proposal Due Date

February 9, 2024

Program Summary