Creation of Operationally Realistic 3D Environment (CORE3D)

A persistent U.S. Government need - for global situational awareness as well as military, intelligence, and humanitarian mission planning - is timely access to geospatially accurate 3D object data. Researchers in academia and industry have attempted to develop ways to capture the 3D geometry and surface properties of objects on the Earth more efficiently, and the U.S. government has also invested in the development of means to exploit 3D data. However, automated methods lag behind manual approaches for creating 3D models with respect to extracting mission relevant information and understanding scenes. While manually constructed models are accurate and reliable, the process to create them is time-consuming and does not satisfy the need for models to support rapid response to military or humanitarian crises in areas where up-to-date models do not exist. The CORE3D program aims to deliver fully automated methods for timely 3D model creation to support our nation’s military, humanitarian and intelligence missions, wherever those missions may be.

High resolution dimensionally accurate models have tended to rely on proximal or direct contact measurements to achieve dimensional fidelity. However, such measurements are not always possible or practical. For example, it is difficult to predict when or where emerging events might occur that may require response informed by an accurate 3D model. With current modeling technology, by the time it is clear that a model may be of assistance, access to the region may be difficult or compromised and the only data available from which to construct a model may bedata from remote sensors. The ability to perform high fidelity 3D GIS modeling using only remote sensor data is essential in order for the U.S. Government to be able to model locations anywhere on Earth. Automated methods to create such models will not only enable their construction in an operationally relevant timeframe for use in times of crisis, but will also enable frequent updating of such models as satellite data are refreshed. Development of methods for construction of high fidelity 3D models fully automatically from remote sensor data is the overall goal of the CORE3D program.

In recent years, the volume of remotely sensed data collected by space and airborne assets has grown at a high rate. In particular, commercial satellites now cover large areas of the Earth with high revisit rates, capture snapshots of geophysical information about the Earth, and deliver high quality imagery in near real-time to customers. The CORE3D program seeks to leverage the spectral, textural, and dimensional information from such satellite data to build 3D models that will not only be dimensionally accurate, but will also enable identification of textures and materials of objects. Construction of high fidelity 3D models fully automatically from satellite data is a specific objective of the CORE3D program.

A further objective of the CORE3D program is to enable the refinement of such models by augmenting satellite data with airborne and GIS vector data, where available. Airborne sensors such as full motion video (FMV), wide area motion imagery (WAMI) and video from unmanned aerial vehicles (UAV) offer the opportunity to improve resolution and fidelity of 3D models but automated methods to exploit these data are limited at present. The CORE3D program will challenge performers to develop methods for automated construction of 3D models from data available across the globe (i.e., satellite data) and to develop methods to refine and improve the models using airborne sensor data which are available from only limited areas of the globe.

With respect to these objectives, IARPA defines “fully automatically” as follows: that the technology to be developed under the CORE3D program will be operational without requiring a “human-in-the-loop”. Methods that require human quality assurance (QA) or quality control (QC) will not meet the program’s objectives and are therefore are out of scope for the CORE3D program.

For the CORE3D program, IARPA seeks to develop methods to use data from space and airborne collections to construct 3D models. It is anticipated that these object models will be fused with existing and volunteered GIS information to geolocate the objects accurately. Accurate geolocation (relative and absolute) of 3D models constructed from remotely sensed data with an accuracy that matches with most accurate input data is a third specific objective of the CORE3D program. In particular, CORE3D aims at geolocation accuracy as good as the accuracy of input data itself.

To meet the overall goal of the CORE3D program and its individual objectives, performers may need to address the following technical challenges:

  • Multi-modal data fusion to include data level, feature level, and decision level fusion
  • Object level segmentation and classification
  • Point cloud generation from multi-view satellite images
  • Representations of complex scene geometry
  • Accurate 3D model fitting/statistical inferencing

Responses to this BAA must clearly explain in detail how the proposed technical approach addresses the specific technical challenges and why the approach may be expected to achieve the specific objectives of the program. Proposals must provide theoretical and/or experimental support for the approaches proposed and must include the development of new algorithms. Proposals that comprise mere integration of existing algorithms will not be considered for funding. Successful approaches will include innovations as well as a realistic plan to develop a prototype system capable of meeting the CORE3D program goals of fidelity with respect to dimension, texture, material and geolocation, as well as taking a reasonable time to construct a high fidelity model automatically.
Collaborative efforts and teaming among potential performers are strongly encouraged. It is anticipated that teams will be multidisciplinary, including but not limited to expertise in such disciplines such as computer vision, remote sensing, machine learning, geology, statistical inference, and photogrammetry. Proposals must demonstrate that the teams contain all the necessary technical expertise and experience for success and that a strong management plan is in place to ensure that milestones are met and that geographically dispersed, multidisciplinary teams will perform in unified fashion to deliver the proposed outcomes.

The CORE3D program is divided into two phases. Phase 1 will last for a period of 27 months. At the end of Phase 1, performer systems will be building models for regions of ~1 km2 in size, and diverse terrain and scene contents. Phase 2 will last 24 months by the end of which performers will be building models ~ 100 km2. As the program progresses, within each Phase, the number of regions to be modeled in each performance period will increase and these regions will be selected from different parts of the world. Proposals submitted in response to this BAA must address both phases of the CORE3D program.

Contracting Office Address

Office of the Director of National Intelligence
Intelligence Advanced Research Projects Activity
Washington, DC 20511

Primary Point of Contact

Hakjae Kim
Program Manager
dni-iarpa-baa-16-06@iarpa.gov

Creation of Operationally Realistic 3D Environment