Computational Cognitive Models of Sensemaking Program
Background & Scope
Intelligence analysts are tasked with identifying relevant information within massive data and synthesizing these fragments into a coherent understanding of the entities, events, and relational networks that characterize a data space. This process of adaptively filtering and interpreting data has been termed "sensemaking" in the literature. However, prevailing models of sensemaking are largely qualitative in nature and fail to reflect emerging insights from cognitive science and neuroscience research regarding the architecture of human information processing. Current models thus provide a shaky foundation upon which to build a deep understanding of analyst cognition. IARPA is soliciting submissions on either one or both of the following topics aimed at addressing these challenges:
1) Cognitive models of sensemaking. IARPA seeks to characterize the challenges and opportunities associated with developing a new class of predictive, computational cognitive models of the intelligence analysis process. Responses should briefly outline a path toward creating a cognitive modeling architecture that:
- Is implemented within a computational framework
- Captures key human sensemaking behaviors, including failure modes
- Is scalable, i.e., is not limited to toy problem sets but rather can perform sensemaking over a large, heterogeneous body of data
- Is grounded in our current understanding of the cognitive architecture of the human brain
IARPA welcomes diverse approaches that combine neural and symbolic modeling methods as well as techniques from the fields of natural language processing, probabilistic learning theory and artificial intelligence. Whatever methods are employed, the overall model architecture should reflect the known functional organization of the human brain/mind. Respondents should also briefly describe the computing resources that will be required to implement their model at scale. IARPA recognizes that page limitations will preclude a detailed description of all elements of the model; thus respondents should strive foremost to explain the general properties of their model, its core computational mechanisms, and its gross functional architecture.
2) Test framework for evaluating sensemaking. IARPA seeks new ideas for a test and evaluation framework to be used in assessing the sensemaking capabilities of artificial systems such as the cognitive models described above. Proposed frameworks should:
- Include a description of the input/output data
- Incorporate graduated levels of difficulty that span a wide range of sensemaking capabilities (from simple model to expert human)
- Include a suite of component tasks that reflect individual sensemaking behaviors
- Describe a grading system, including a plan for benchmarking model performance against human performance
Respondents may submit responses to topic 1, topic 2, or both. If submitting to both topics, respondents should prepare two separate submissions. Each submission must adhere to the formatting guidelines described below, including page limitations.
The responses to this RFI will be used to help in the planning of a one and a half day workshop on computational cognitive models of sensemaking, the results of which may justify a multi-year competitive program. The selection of topics and setting of the agenda of this workshop will in part be informed by the responses. It is anticipated that this workshop will be held in July, 2009.
Preparation Instructions to Respondents
IARPA solicits respondents to submit ideas related to this topic for use by the Government in formulating a potential program. IARPA requests that submittals briefly and clearly describe the potential approach or concept, outline critical technical issues, and comment on the expected performance, robustness, and estimated cost of the proposed approach. This announcement contains all of the information required to submit a response. No additional forms, kits, or other materials are needed.
IARPA appreciates responses from all capable and qualified sources from within and outside of the US. Because IARPA is interested in an integrated approach, responses from teams with complementary areas of expertise are encouraged. Responses have the following formatting requirements:
- A one page cover sheet that identifies the title, organization(s), respondent's technical and administrative points of contact - including names, addresses, phone and fax numbers, and email addresses of all co-authors, and clearly indicating its association with IARPA-RFI-09-02;
- A substantive, focused, one-half page executive summary;
- A description (limited to 5 pages in minimum 12 point Times New Roman font, appropriate for single-sided, single-spaced 8.5 by 11 inch paper, with 1-inch margins) of the technical challenges and suggested approach(es);
- A list of citations (any significant claims or reports of success must be accompanied by citations, and reference material MUST be attached);
- Optionally, a single overview briefing chart graphically depicting the key ideas.
Disclaimers and Important Notes
This is an RFI issued solely for information and new program planning purposes and does not constitute a solicitation. Respondents are advised that IARPA is under no obligation to acknowledge receipt of the information received, or provide feedback to respondents with respect to any information submitted under this RFI.
Responses to this notice are not offers and cannot be accepted by the Government to form a binding contract. Respondents are solely responsible for all expenses associated with responding to this RFI. It is the respondents' responsibility to ensure that the submitted material has been approved for public release by the organization that funded whatever research is referred to in their response.
The Government does not intend to award a contract on the basis of this RFI or to otherwise pay for the information solicited, nor is the Government obligated to issue a solicitation based on responses received. Neither proprietary nor classified concepts or information should be included in the submittal. Input on technical aspects of the responses may be solicited by IARPA from non-Government consultants/experts who are bound by appropriate non-disclosure requirements.
For information contact:firstname.lastname@example.org