Knowledge Discovery and Dissemination (KDD)

Intelligence analysts must gather and analyze information from a wide variety of data sets that include: general references, news, technical journals and reports, geospatial data, entity databases, internal reports and more. The different terminologies, formats, data models, and contexts make it difficult to perform advanced analytic tasks across different data sets.

If there are only a small, fixed number of data sets involved in an intelligence problem, then it may be practical to map all of the data sets to a common data model and to develop specialized analytic tools tailored to the problem. However, if the problem changes over time, the data sets are large or numerous, or there are new data sets that need to be integrated with those already in use, then a new approach is required. The focus of the KDD program is to develop novel approaches that will enable the intelligence analyst to effectively derive actionable intelligence from multiple, large, disparate sources of information, to include newly available data sets previously unknown to the analyst.

The ability to quickly produce actionable intelligence from unanticipated, multiple, varied data sets require research advances in two key areas: (1) alignment of data models; and (2) advanced analytic algorithms. Making advances in these two research areas, and fully characterizing the performance of the research results using real Intelligence problems, is the focus of the IARPA Knowledge Discovery and Dissemination (KDD) Program. Performers shall perform research in both areas and develop prototype systems that implement their techniques and research results. The KDD Program will provide data sets to support research and development in addition to extensive test and evaluation.

KDD test and evaluation will take place on an annual cycle, with each performer applying their prototype systems to challenge problems defined by the KDD Program. KDD evaluation of prototype systems will take place at government facilities and will use realistic Intelligence problems and real Intelligence data. The research supported by KDD will generally be unclassified, but the annual KDD evaluations will involve data sets classified no higher than SECRET//NOFORN.

The KDD Program requires a combination of innovative research and the capability to develop robust prototypes. Research goals should be set, and research plans should be made, to take full advantage of the length of the KDD Program. The KDD Program expects a staged approach to prototype development; each successive prototype will leverage research progress made since the previous prototype. IARPA encourages teaming between academic and commercial entities to leverage the strengths of both types of organization. KDD requires the prime contractor for each performer team to have personnel and a facility cleared at the SECRET//NOFORN level at the time their proposal is submitted.

KDD is planned as a 51-month program and anticipates making multiple awards.


For information contact: