Knowledge Representation in Neural Systems (KRNS) BAA Questions

#QuestionAnswerDate Posted
001 How many awards will be made, and what is the expected size of each award? See IARPA Frequently Asked Questions (FAQ) B1 and B2. FAQs are found at http://www.iarpa.gov/faq.html. 01/16/13
002 Are foreign nationals eligible to participate? As stated in BAA Section 3.A Eligible Applicants, “Foreign participants and/or individuals may participate to the extent that such participants comply with any necessary Non-Disclosure Agreements, Security Regulations, Export Control Laws and other governing statutes applicable under the circumstances.” 01/16/13
003 Will Phase 1 Development Concepts be single words or sentences? Development Concepts in Phase 1 will be in the form of simple sentences, similar to the Test Concepts (see BAA Section 1.A.5). 01/31/13
004 Regarding Phase 1, Task 1:

(a) Will there always be multiple Target Concepts per Test Concept?

(b) In the case where the Test Concept consists of Target Concepts A, B and C, will there ever be a situation where the system is required to rank Target B highly but not Targets A or C?
(a) Yes.

(b) No. For a given Test Concept, the goal should always be for the performer's system to rank all of the constituent Target Concepts highly within the sorted list (see BAA Section 1.B.1 for details).
01/31/13
005 Regarding Phase 1, Task 2:
  1. (a) How many Target Concepts will there be?

  2. (b) Will the same Target Concept be found in multiple Test Concepts?

  3. (c) If so, how should those Test Concepts be ranked?
(a) The number of Target Concepts used for Phase 1, Task 2 will be specified at Program Kickoff.

(b) Yes.

(c) In such case, the system should aim to rank those Test Concepts in order of their semantic relatedness to the Target Concept as behaviorally reported by the subject (see BAA Section 1.B.1 for details).
01/31/13
006 Is it correct to assume that “cost or pricing data” will not be required even for a procurement contract award > $700,000 as the BAA will generate adequate price competition precluding this requirement? Adequate price competition is anticipated for this acquisition. Therefore, an exception exists to the requirement to submit Certified Cost and Pricing data at this time, as specified in the Federal Acquisition Regulation (FAR) 15.403-1(b). However, the Government reserves the right to request at a later date, Certified Cost and Pricing Data or Other than Certified Cost and Pricing Data in order to conduct a determination on realism and reasonableness. Interested offerors are still required to submit supporting cost and pricing information in sufficient detail to substantiate the summary cost estimates in the Technical Proposal volume as it relates to cost information as set forth in the subject BAA solicitation Section 4.B.2. 01/31/13
007 Are Prime Contract Cost Proposals required to follow the exact template provided in the Prime Contractor Cost Element Sample (Appendix E) of the BAA, or is the Sample intended to be used only for guidance purposes? The Government prefers that the offeror use the exact template in Appendix E. If a different template is used, it should contain all of the information requested in Appendix E. 02/08/13
008 The period of performance is outlined as Phase 1a (Base); Phase 1b (Option) and Phase 2 (Option). The cover sheets have the periods broken down by: Base Period, Option Period 1, and Option Period 2. Does Option Period 1 correspond to Phase 1b (Option) and Option Period 2 correspond to Phase 2 (Option)? Yes. 02/08/13
009 Regarding Volume 1, Proposal Content, Section 2, Summary of Proposal:

(a) Is Section 2 limited to 2 pages?

(b) If so, is it acceptable to add crossreferences that refer to Section 3? Specifically would it be acceptable to omit information that is being requested in Section 2 (i.e. Project Schedule, Cost Schedule Table, Key Personnel table, Organizational Chart, etc.) and place it in Section 3 only?
(a) No. Amendment 1 of the BAA increases the page limit from 2 to 5 pages.

(b) Cross‐references are acceptable; however, crossreferences alone are not a sufficient substitute for the requested information.
02/08/13
010 Is it possible to meet with the Program Manager to discuss our proposal ideas? See Question P5 of IARPA Frequently Asked Questions (FAQ). FAQs are found at http://www.iarpa.gov/faq.html. 02/08/13
011 Is a current employee of a UARC or FFRDC eligible to submit a proposal if that person resigns his or her position prior to the submission date? See Questions P1 and P3 of IARPA Frequently Asked Questions (FAQ). FAQs are found at http://www.iarpa.gov/faq.html. For more information on IARPA’s approach to managing Organizational Conflicts of Interest (OCI), see: http://www.iarpa.gov/IARPA_OCI_081809.pdf. 02/08/13
012 Is an employee of a university that hosts a UARC eligible to submit a proposal? See Question P1 and P6 of IARPA Frequently Asked Questions (FAQ). FAQs are found at http://www.iarpa.gov/faq.html. For more information on IARPA’s approach to managing Organizational Conflicts of Interest (OCI), see: http://www.iarpa.gov/IARPA_OCI_081809.pdf. 02/08/13
013 Is a person who is employed part‐time by an ineligible organization able to submit a proposal if he or she does so via an eligible organization? See Questions P1 and P3 of IARPA Frequently Asked Questions (FAQ). FAQs are found at http://www.iarpa.gov/faq.html. For more information on IARPA’s approach to managing Organizational Conflicts of Interest (OCI), see: http://www.iarpa.gov/IARPA_OCI_081809.pdf. 02/08/13
014 Could you provide a general description of the expected breadth of the Development/Test concepts in terms of the knowledge that will be required for comprehension? As stated in BAA Appendix I (Description of Concepts), “concepts in KRNS will be chosen that are representative of general human knowledge, perceptions, actions, and interactions with events and objects in the world.” Thus, “all individual concepts that appear in KRNS will be familiar to a typical adult, and would appear in corpora of written American English with reasonable frequency.” In addition, “obscure, esoteric, unfamiliar, uncommon jargon or slang, or offensive concepts are out of scope.” See Appendix I for additional descriptions and examples of KRNS concepts. 02/08/13
015 With respect to cohorts, is it expected that the selected cohorts will maximize variance (across cohorts) in a general way with respect to the theory or would it be acceptable to have two cohorts that differ in a specific aspect? IARPA recognizes that differences between cohorts may manifest themselves in a specific aspect /brain area(s) or in a general way across many brain areas; thus, either approach could be acceptable. Please note that offerors should provide a clear justification of their choice of cohorts (as stated in BAA Section 1.A.4.iii.A, Subject Recruitment Protocol). 02/08/13
016 Could you clarify the expected resolution of the predicted neural activity patterns? As described in BAA Section 1.A.4.ii (Implementation), the predicted neural activity patterns will be generated using the same predictive model used to perform the interpretation tasks (described in BAA Section 1.B.1). Thus, the spatial and temporal resolution of the predicted activity patterns will be determined by whatever resolution is required to successfully perform the interpretation tasks (where “success” is defined as achieving the metrics specified in BAA Table 1). 02/08/13
017 How many Development Concepts will there be in Phases 1 and 2, roughly? The number of Development Concepts for each phase will be specified during the course of the program. However, it is anticipated that the number of Development Concepts will be slightly greater than the number of Test Concepts. 02/11/13
018 Given that the Test Concepts for Phase 1a are provided at Kickoff (as an unspecified subset of the Development Concepts), does this imply that the performer can begin collecting Phase 1a neural test data as soon as all the necessary approvals for human subject research have been obtained? Yes. As stated in footnote 19 (BAA Table 2, Program Timeline), “Although Month 11 marks the official beginning of the Phase 1a data collection period, performers are free to utilize for testing purposes any data collected for the Test Concepts prior to this point.” 02/11/13
019 With respect to Phase 1a, would it be possible for an offeror to propose to:

(a) collect neural data for all Development Concepts, at least for a few subjects?

(b) experiment with different regimes for collecting neural data to see which one produces better signal before the Month 11 date (where the actual Test Concepts are specified)?
  1. (a) Yes.

(b) Yes. Experimentation with different data collection regimes is consistent with the notion of “ongoing research” (BAA Section 1.A.1) and “ongoing experimentation” (BAA Figure 1, Section 1.A.5).
02/11/13
020 (a) Is there a possibility that the required numbers of Test Concepts in Phase 1b or 2 will be lowered?

(b) Is it possible for an offeror to propose imaging fewer than the number of Test Concepts stated in the BAA if the offeror believes that the result would be higher quality data?
(a) Yes, it is possible that during the course of the program the number of Test Concepts could be lowered if the Government determines that robust results can be obtained with fewer data. The numbers of Test Concepts stated in the BAA are “approximate” (BAA Section 1.A.5, Test Concepts) and represent an upper bound on the total amount of neural test data required (as indicated by the use of the phrase “up to” in BAA Table 1).

(b) Offerors should propose a data collection plan consistent with the schedule and numbers of Test Concepts stated in the BAA. However, as part of their discussion of the risks associated with the BAA’s aggressive data collection timeline (see BAA Section 1.A.4.iii.C), offerors may discuss the expected tradeoff between the number of Test Concepts imaged and the performance of their system.
02/11/13
021 BAA Section 4.B.1.2, Paragraph F (Project contributors) states that participants are expected to dedicate no less than 10% of their time, whereas BAA Section 4.B.1.3, Paragraph I (Detailed Management Plan) states that participants are expected to dedicate at least 20% of their time. Is there a discrepancy? No. The 10% figure stated in BAA Section 4.B.1.2, Paragraph F (Project contributors) refers to any contributor (including consultants), whereas the 20% figure stated in BAA Section 4.B.1.3, Paragraph I (Detailed Management Plan) refers specifically to participants who are identified in the proposal as ”key personnel” or ”significant contributors.” 02/21/13
022 BAA Section 4.B (Volume 1) lists Attachment 6 as “Consultant Commitment Letters,” but BAA Section 4.B.1.4 (Section 4: Attachments) refers to Attachment 6 as “Draft Human Subjects Protocol.” Could you clarify what Attachment 6 should be? This discrepancy is resolved in Amendment No. 2 of the BAA, which identifies Consultant Commitment Letters as Attachment 6 and Draft Human Subjects Protocol as Attachment 7. 02/21/13
023 Are offerors limited to recruiting just two cohorts? No. As stated in the BAA section 1.A.4.iii.A (Subject Recruitment Protocol), "Offerors may propose more than two cohorts provided that it is feasible to collect neural imaging data for all subjects within the schedule constraints specified in Table 2." 02/21/13
024 Are there any restrictions in defining subject cohorts (apart from maximizing inter-group variance and minimizing intra-group variance)? No. However, as stated in BAA 1.A.4.iii.B (Concept Elicitation Protocol), “In order to enable comparisons across individual subjects and cohorts, the same concept should be elicited the same way (i.e., via the same protocol and same stimuli) for all subjects.” This in turn may impose practical constraints on how cohorts are defined and selected, as described in BAA footnote 5. 02/21/13
025 (a) Are offerors permitted to incorporate additional words (e.g., nonsense words) and/or phrases and/or sentences into the T&E-provided Development Concepts?

(b) Are offerors permitted to alter the Test Concepts, e.g., by including/removing words and phrases?
(a) Yes. Offerors are free to use the T&E-provided Development Concepts as they see fit, including altering/augmenting the set of Development Concepts with concepts of their own choosing. As stated in BAA Section 1.A.4.ii (Implementation), “If any sets of concepts (in addition to the Development Concepts provided by the T&E Team) are to be used in system development, these should be described.” Offerors should note, however, that for Phase 1a, Test Concepts will be selected from the original set of T&E-provided Development Concepts (as stated in BAA Section 1.A.5).

(b) No.
02/21/13
026 Recent amendments to the BAA have extended the maximum length of the executive summary section of the proposal from two pages to five pages. Is there a concomitant change in the total proposal page count? No. Per BAA Section 4.B.1, offerors must limit the technical volume (Volume 1) of their proposals to 30 pages. 02/21/13
027 If we envision that the experiments we have in mind will be covered under existing IRBs of the participating labs:

(a) Can we assume that Government Human Research Protection Officer (HRPO) approval would happen one or two months earlier, as per BAA timeline?

(b) Can we scan pilot subjects at our own cost until HRPO approval is obtained?
(a) Per BAA Table 2 (Program Timeline), offerors should allow four months for the HRPO approval process to be completed starting from the date of submission of the Human Subject Research (HSR) documentation package. Submission of the HSR package may occur earlier than Month 2 (as specified in BAA Table 2) assuming that all local IRB approvals have been obtained.

(b) Any and all research involving the use of human subjects sponsored by, contracted for, or conducted on behalf of IARPA must be in accordance with guidelines issued by the Department of Health and Human Services. Therefore, KRNS performers must obtain all HRPO approvals prior to any scanning of pilot subjects.
02/26/13
028 Can a fraction of research assistant time be included in the budget given the expected large number of scanning sessions? It is at the discretion of the offeror to determine an appropriate team composition and allocation to best meet the program objectives. 02/26/13
029 Is it allowed to submit the proposal in pdf? Yes. As stated in BAA Section 4.C.2 (Proposal Delivery), “offerors should upload proposals, including Volume 1, Volume 2, scanned certifications and permitted additional information in ‘pdf’ format.” 02/26/13
030 Regarding BAA Section 4.B.1.3 (Section 3: Detailed Proposal Information):

(a) In Paragraph B (A detailed description of the objectives…), what is the difference between "scientific relevance" and "expected significance?”

(b) With respect to Paragraph D (Data sources), is it necessary to elaborate on the experimental protocol if this is described in detail elsewhere in the proposal? Or is it sufficient to focus on the data sources and analysis techniques?

(c) With respect to Paragraph I (Detailed Management Plan), can the same biographical sketches be used as in the Summary of Proposal?
(a) "Scientific relevance" concerns the relevance of the proposed work with respect to the scientific goals of the KRNS Program, whereas "expected significance” refers to the overall scientific and technological significance of the work.

(b) Paragraph D pertains to “human subject populations, data sources, and data analysis techniques.” A detailed description of the experimental protocol is not required under Paragraph D; rather, this information is required under Paragraph B, which states that offerors “should clearly detail the technical method(s) and/or approach(es) that will be used.”

(c) Biographical sketches are to be included in the Detailed Management Plan. Per IARPA-BAA-12-05 Amendment 01, biographical sketches are no longer required in the Summary of Proposal.
02/28/13
031 Who is responsible for creating the experimental task to be performed by subjects during neural imaging: the T&E Team or the offerors? As discussed in BAA Section 1.A.4.iii.B (Concept Elicitation Protocol), offerors are responsible for designing the experimental task. 02/28/13
032 BAA Section 4.B.1.2, Paragraph F (Project contributors) requires a specification of the level of effort by participants as a fraction of time or number of hours. If a participant is involved in only part of the effort (e.g., a particular period of a particular phase), how should this be indicated – i.e., should time be amortized over the entire duration of a phase or should it be reported as a number of hours? It is the responsibility of the offeror to best depict their proposed solution and therefore the discretion of the offeror on how to best convey the allocation of resources, participation of teammates, deliverable schedule and cost, etc. 02/28/13
033 BAA Section 4.B.1.2, Paragraph C (Schedule and milestones…) requests an "estimate of cost for each deliverable.” In computing this estimate, is it acceptable to aggregate the time in months of each task in the Gantt chart (BAA Section 4.B.1.2, Paragraph F) up to the deliverable, and then multiply by the cost per month of each person involved in that task? It is the responsibility of the offeror to best depict their proposed solution and therefore the discretion of the offeror on how to best convey the allocation of resources, participation of teammates, deliverable schedule and cost, etc. 02/28/13