Camera Network Research Data Collection

The Intelligence Advanced Research Projects Activity is seeking information on research efforts in the area of computer vision within multi-camera video networks. This request for information is issued solely for information gathering and planning purposes; this RFI does not constitute a formal solicitation for proposals. The following sections of this announcement contain details of the scope of technical efforts of interest, along with instructions for the submission of responses.

Background & Scope

This RFI seeks capability statements relating to the collection of research data from multi-camera video networks in support of computer vision research. Over the past five years, there have been notable advances in computer vision approaches to facilitate tracking and re-identification of persons in security camera networks. However, the primary datasets available to the research community for algorithm training and performance evaluations, while incredibly valuable, are somewhat limited in subject count, camera network scope, and environmental factors, resulting in a disconnect between the data being leveraged by researchers and the types of video data that would exist in actual video networks utilized by public safety and law enforcement entities. Further research in the area of computer vision within multicamera video networks may support post-event crime scene reconstruction, protection of critical infrastructure and transportation facilities, military force protection, and in the operations of National Special Security Events.

This RFI seeks approaches, capabilities, and previous experience related to the ability to carry out a potential video research data collection with the following characteristics:

  • Data from a large camera network containing a minimum of twenty (20) video feeds of varying positions, views, resolutions, and frame rates with both overlapping and non-overlapping fields of view;
  • Data captured over a large region (~10,000 sq. meters) with multiple intersections, paths for pedestrian foot traffic, and buildings entrances/exits;
  • Data collected in an urban or semi-urban environment with distractors and occlusions representative of real-world conditions, such as signs, vehicles (parked and moving), trees, and other obstructions;
  • Data collected over multiple days with varying illumination and weather conditions;
  • Data involving a minimum of 5,000 pedestrians;
  • Data involving a minimum of 200 subject volunteers injected into the camera network view and given instructions on how to behave and/or where to go in the camera network;
  • Annotations for all video footage detailing ground-truth bounding boxes of each person in each frame of each camera view to facilitate research, development, test, and evaluation of person tracking and re-identification algorithms;
  • Minimum video footage for the entire dataset totaling an estimated 960 hours (20 cameras x 12 hrs/day x 4 days);
  • A dataset that is permissible to be released to the general research community under a privacy, legal, and policy approved data release process.

Responses to this RFI should answer any or all of the following questions:

  1. What is your organization’s proposed approach to conducting the described data collection? Where would the collection take place? Whose camera network would be utilized (e.g., new network stood up by research team, existing partner organization’s network)? If an existing camera network is proposed, are they third-party business-owned, local government-owned, university-owned, your organization’s cameras, other? What would be the plan for properly administering and receiving approval for human subject research (HSR)? Be sure to address both the non-interacted pedestrians and the scripted volunteers, and the environment/location the proposed network would be deployed?
  2. How long is estimated for the planning, collection, and annotation stages of the effort? How would data management and label annotation be conducted to ground-truth the data in an efficient (resource and cost) manner while maintaining scientific accuracy? Describe the data protection, handling, and management plan for a collection of this nature and scope.
  3. What experience does your organization have in collecting video data for research purposes? What was the purpose and nature of the dataset(s)? Describe the size, complexity, and scope of the video data collection. What types of annotations were applied and how? Describe logistics and IRBinteractions necessary to plan and prepare for the human subject research collection? Did past collections occur in public or semi-public spaces requiring coordination with local facilities or governing organizations/offices, if so, describe the logistics and coordination involving the space and partners? Does your organization have any experience in capturing video data in a setting where conspicuous signage was used to notify pedestrians as opposed to only relying on researchermediated informed HSR consent?
  4. What research experience does your organization have in the field of computer vision, such that you can demonstrate an understanding of the types of data, research challenges, and annotation practices needed to leverage the data from a large multi-camera network?
  5. Carrying out the data collection described may require partnering with a state, local, or municipal government agency or an outside third party organization to utilize cameras or collection spaces under their jurisdiction. To that extent, what are some previous experiences or demonstrated partnerships that may be utilized to realize this data collection? Are there any letters of support from related organizations that can be provided that would support a future potential data collection as described above?
  6. How does your organization plan to address any legal and/or privacy issues that may be raised by your proposed data collection approach?
  7. Would your organization consider carrying out this data collection if doing so meant that it would be prohibited from participating as a performer in a specific future IARPA research effort that utilized this data for test and evaluation?

Preparation Instructions to Respondents

IARPA requests that respondents submit capabilities and previous experiences related to this topic for use by the Government in formulating a potential program. If appropriate, respondents may also choose to provide a non-proprietary rough order of magnitude estimate regarding what such a data collection might require in terms of funding and other resources. This announcement contains all of the information required to submit a response. No additional forms, kits, or other materials are needed.

IARPA appreciates responses from all capable and qualified sources from within and outside of the US. Responses have the following formatting requirements:

  1. A one page cover sheet that identifies the title, organization(s), respondent's technical and administrative points of contact - including names, addresses, phone and fax numbers, and email addresses of all co-authors, and clearly indicating its association with RFI-19-06;
  2. A substantive, focused, one-half page executive summary;
  3. A description (limited to 5 pages in minimum 12 point Times New Roman font, appropriate for singlesided, single-spaced 8.5 by 11 inch paper, with 1-inch margins) addressing the questions above and any other relevant topics;
  4. A list of citations (any significant claims or reports of success must be accompanied by citations);
  5. Optionally, letter(s) of support from other organizations or state, local, or tribal government agencies summarizing willingness to assist or participate in the described data collection;
  6. Optionally, a single overview briefing chart graphically depicting the key ideas;
  7. Optionally, a ROM cost estimate regarding what such a data collection might require in terms of funding and other resources. It is recommended that the ROM separate the funding/resource estimates into those needed for data collection and annotation labeling and address what effect increasing/decreasing the scale of the collection by 25% would have on the ROM.

Submission Instructions to Respondents

Responses to this RFI are due no later than 12:00 p.m., Eastern Time, on May 10, 2019. All submissions must be electronically submitted to as a PDF document. Inquiries to this RFI must be submitted to Do not send questions with proprietary content. No telephone inquiries will be accepted.

Disclaimers and Important Notes

This is an RFI issued solely for information and planning purposes and does not constitute a solicitation. Respondents are advised that IARPA is under no obligation to acknowledge receipt of the information received, or provide feedback to respondents with respect to any information submitted under this RFI.

Responses to this notice are not offers and cannot be accepted by the Government to form a binding contract. Respondents are solely responsible for all expenses associated with responding to this RFI. IARPA will not provide reimbursement for costs incurred in responding to this RFI. It is the respondent's responsibility to ensure that the submitted material has been approved for public release by the information owner.

The Government does not intend to award a contract on the basis of this RFI or to otherwise pay for the information solicited, nor is the Government obligated to issue a solicitation based on responses received. Neither proprietary nor classified concepts or information should be included in the submittal. Input on technical aspects of the responses may be solicited by IARPA from non-Government consultants/experts who are bound by appropriate non-disclosure requirements.

Contracting Office Address:

Office of the Director of National Intelligence
Intelligence Advanced Research Projects Activity
Washington, District of Columbia 20511
United States

Primary Point of Contact:

Dr. Lars Ericson
Intelligence Advanced Research Projects Activity


Posted Date: March 26, 2019
Responses Due: May 17, 2019