Lessons Learned Knowledge Management
The Intelligence Advanced Research Projects Activity (IARPA) is seeking information about organizational lessons-learned analyses and knowledge management processes. This request for information (RFI) is issued solely for information gathering and planning purposes; this RFI does not constitute a formal solicitation for proposals. The following sections of this announcement contain details of the scope of technical efforts of interest, along with instructions for the submission of responses.
Background & Scope
Lessons Learned Knowledge Management (LLKM) is defined as the recording of experience-based lessons-learned analyses such that individuals can discover, retrieve, and apply lessons appropriate to their current circumstances. Lessons-learned analysis includes retrospective analysis of successes and failures, counterfactual analysis of alternative actions and likely consequences, and hypothetical analysis to recommend actions in similar circumstances.
IARPA is specifically interested in innovative LLKM methods that:
- Yield accurate lessons about the causes for successes and failures, forecasts about what would have occurred if different actions had been taken, and forecasts of outcomes that will occur in future situations where the lessons are applied.
- Mitigate bias in determining the lesson to be learned from experience (e.g., judgment and memory biases, such as fundamental attribution errors, hindsight memory bias), enhance counterfactual reasoning of what would have occurred (e.g., formal logics for counterfactual inference, statistical methods for counterfactual analyses, psychological research in counterfactual thinking and counterfactual analysis methods used by historians), or adapt rigorous case-study methods to lessons-learned analysis (e.g., formal case study methods used in scientific inquiry, case studies in business analysis, accident investigations, etc.).
- Test and evaluate whether experience-based lessons-learned methods yield accurate lessons (e.g., simulation-based testing, short-term forecasting), including metrics to assess the accuracy of identified causes.
- Address the KM recording of lessons in a manner that facilitates the discovery, retrieval, and application of lessons that are appropriate for future circumstances by individuals who were not participants in the original learning experience (e.g., use a structured argument to justify a lesson and then re-use the same structure argument to determine if applying the lesson in a new situation is justified.)
- Test the efficacy of the KM component of an LLKM system, including metrics to assess the selection of appropriate recorded lessons and their application.
Note that IARPA is interested in finding methods to learn and apply accurate lessons. For this RFI, IARPA is not interested in methods that facilitate organizations' adoption and dissemination of lessons.
Responses to this RFI should answer any or all of the following questions:
- What are existing methods for lessons-learned analysis? How do they attempt to separate accurate vs. inaccurate lessons?
- What are novel approaches that could improve the accuracy and efficacy of lessons-learned analyses?
- Are there existing approaches to knowledge management that support capturing lessons-learned knowledge? How do they encode learned-lessons? How do they attempt to ensure that relevant lessons are discovered and appropriately reused in future circumstances?
- What are novel approaches to improving KM that could improve encoding, discovery, and appropriate reuse of lessons-learned knowledge?
- What are possible test domains, test protocols, and metrics that could be used to estimate the extent to which a lessons-learned analysis yields accurate lessons?
- What are possible test domains, test protocols, and metrics that could be used to estimate the extent to which a KM process results in appropriate reuse of lessons-learned knowledge?
Preparation Instructions to Respondents
IARPA requests that respondents submit ideas related to this topic for use by the Government in formulating a potential program. IARPA requests that submittals briefly and clearly describe the potential approach or concept, outline critical technical issues/obstacles, describe how the approach may address those issues/obstacles and comment on the expected performance and robustness of the proposed approach. If appropriate, respondents may also choose to provide a non-proprietary rough order of magnitude (ROM) regarding what such approaches might require in terms of funding and other resources for one or more years. This announcement contains all of the information required to submit a response. No additional forms, kits, or other materials are needed.
IARPA appreciates responses from all capable and qualified sources from within and outside of the US. Because IARPA is interested in an integrated approach, responses from teams with complementary areas of expertise are encouraged.
Responses have the following formatting requirements:
- A one page cover sheet that identifies the title, organization(s), respondent's technical and administrative points of contact - including names, addresses, phone and fax numbers, and email addresses of all co-authors, and clearly indicating its association with RFI-14-01;
- A substantive, focused, one-half page executive summary;
- A description (limited to 5 pages in minimum 12 point Times New Roman font, appropriate for single-sided, single-spaced 8.5 by 11 inch paper, with 1-inch margins) of the technical challenges and suggested approach(es);
- A list of citations (any significant claims or reports of success must be accompanied by citations, and reference material MUST be attached); Optionally, a single overview briefing chart graphically depicting the key ideas.
Disclaimers and Important Notes
This is an RFI issued solely for information and planning purposes and does not constitute a solicitation. Respondents are advised that IARPA is under no obligation to acknowledge receipt of the information received, or provide feedback to respondents with respect to any information submitted under this RFI. Responses to this notice are not offers and cannot be accepted by the Government to form a binding contract. Respondents are solely responsible for all expenses associated with responding to this RFI. IARPA will not provide reimbursement for costs incurred in responding to this RFI. It is the respondent's responsibility to ensure that the submitted material has been approved for public release by the information owner.
The Government does not intend to award a contract on the basis of this RFI or to otherwise pay for the information solicited, nor is the Government obligated to issue a solicitation based on responses received. Neither proprietary nor classified concepts or information should be included in the submittal. Input on technical aspects of the responses may be solicited by IARPA from non-Government consultants/experts who are bound by appropriate non-disclosure requirements.
For information contact:firstname.lastname@example.org
IARPA-RFI-14-01 CLOSEDPosted Date: December 3, 2013
Responses Due: March 14, 2014