Strengthening Human Adaptive Reasoning and Problem-solving (SHARP) BAA Questions

#QuestionAnswerDate Posted
001 Are you able to give us any information on: 1) The approximate number of awards you plan to make? 2) Approximate annual budget (direct or total cost) per year for the 3.5 year program? The answers to these questions can be found in the IARPA Frequently Asked Questions (FAQ), which can be accessed at http://www.iarpa.gov/faq.html. 06/12/13
002 In the overview on fbo.gov it is indicated that “Despite some promising results, however, there are methodological and practical shortcomings that currently limit the direct applicability of this research for the Intelligence Community.” Does this mean that the proposed project should address these shortcomings and should only involve very similar participants to members of the Intelligence Community? As described in BAA Section 1.A.3.ii, “SHARP intervention effects are expected to generalize to an IC analyst population that is diverse in educational and professional background and includes a substantial proportion of highly educated and/or high-cognitive ability personnel. They range in experience from less than a year to more than 30 years on the job, with degrees in a wide range of disciplines, including economics, linguistics, law, international policy, science, mathematics, and engineering. Offerors must describe how findings from their proposed participant samples will plausibly generalize to the IC workforce. 06/12/13
003 Can you clarify the distinction between cohorts and groups? How many subjects would you like in the study? ** Please note this answer was revised on July 1, 2013.

During each phase, there will be two distinct cohorts (internal and T&E), each consisting of two groups (intervention and control) as stated in BAA Section 1.A.3.ii.

As stated in Section 1.A.5, page 13 in Amendment 02 of the BAA, “…at a minimum offerors should plan to test 110 subjects per group (interventions(s) and controls) on the T&E ARP outcome measure." This is the approximate number of subjects per group that offerors should plan to test for the T&E cohort.

The BAA does not include any requirements on the number of subjects to be tested in the internal cohort. As stated in BAA Section 1.A.3.vi., “The proposal must include a detailed power analysis, including consideration of sample characteristics, attrition, approaches to tailor the intervention(s) in Phase 2, and other relevant factors.”
06/12/13
004 Phase 1 results from the internal cohort are due at month 11, while results from the T&E cohort are due at month 22. Does that imply that during phase 1, it is permissible to recruit and test the internal cohort first, and to recruit and test the T&E cohort second, as long as recruitment methods and participant demographics are consistent across the two cohorts? It is at the discretion of the offeror to best propose a solution that meets the conditions defined in BAA Section 1.A.5. 06/18/13
005 Is it true that the intervention(s) cannot change between the testing of the internal cohort and the testing of the T&E cohort in phase 1? As stated in BAA Section 1.A.3.ii., “The cohorts should be identified, recruited, assigned to conditions, and treated in identical fashion, with the only differences being the ARP measurement and the metrics associated with these measurements.” 06/18/13
006 Would it be allowable to test several intervention configurations of a multi-component treatment on the internal cohort (e.g. intervention A alone versus intervention B alone versus intervention A+B, compared to active and non-active controls) during Phase 1a, and then to complete Phase 1b testing of the T&E cohort comparing only the intervention configuration found to work the best in the internal cohort (e.g. intervention A+B) to the active and non-active controls? It is at the discretion of the offeror to propose a solution that best meets the conditions defined in BAA Section 1.A.3.ii. (see the answer to Question #005). 06/18/13
007 Is it permissible to perform pilot testing to tune the intervention(s) prior to the start of testing in phase 1? See the answer to question #006. Please note that, as stated in BAA Section 6.B.5, page 42, “No IARPA funding can be used towards human-subject research until ALL approvals are granted.” 06/18/13
008 There was mention of 240 participants in Table 2. Is this the recommended number of subjects per group? The examples listed in Table 2: Sample Waypoint Table on page 18 of the BAA are only included to illustrate the type of information that should be filled in under each category. Please see the answer to Question #003 for more information on number of subjects. 06/18/13
009 The BAA does not specify that the participants have completed undergraduate degrees, just that the population should be relevant to the IC analyst population. Does the specification from the Proposer’s Day slides still hold – that all participants should have completed undergraduate degrees – or has this been changed? As specified in Section 1.A.3.ii, the IC workforce has personnel with a “range in experience from less than a year to more than 30 years on the job, with degrees in a wide range of disciplines.” As stated in the same section, “Offerors must describe how findings from their proposed participant samples will plausibly generalize to the IC workforce.” 06/18/13
0010 The BAA mentions local IRB review, but no government IRB review. Does that mean that the protocol will not need to go through government IRB review (such as a Human Research Protection Office) or is that still undetermined? As stated in BAA Section 1.A.5, page 12, the local IRB approval of a performer’s research protocol will undergo “a review and approval from the SHARP Government Contracting Agent,” which involves verifying that the local IRB approvals were completed in accordance with current Government regulations. This verification will be conducted by a non-Department of Defense Government Contracting Agent. 06/18/13
011 Would a proposed intervention that uses hardware, which is currently cumbersome to deploy outside a lab but will have portable consumer device prototypes, be considered in-scope? Also, will the anticipated ease for future fielding of the envisioned intervention be a factor in evaluating proposals? Please review BAA Sections 1.A.3.i., “Interventions,” 1.A.4. “Out of Scope,” and 5.A. “Evaluation Criteria.” As stated in BAA Section 1.A.3.i.: “As SHARP research is meant to be applicable for modern work places and personnel, interventions cannot require more than a total of 210 minutes of active participation in any given calendar week.” 06/18/13
012 Would some testing of supplemental treatments/interventions for the purpose of causal model development/validation, in addition to testing of the main intervention, be considered in-scope? As stated on page 2 of the BAA, while one of the goals of the SHARP program is “developing evidence-based causal models, which will both advance the theoretical bases for the proposed intervention(s) as well as assist in tailoring the proposed intervention(s) for individuals in order to optimize benefits,” testing and validating causal models independent of proposed intervention(s) is not a primary goal of the program. 06/18/13
013 Do we need to include a cost estimate to conduct the intervention across both cohorts combined? Yes, as stated in BAA Section 1.A.3.ii., “Performers will be responsible for recruiting, screening, and enrolling subjects, in addition to conducting the intervention(s), and administering tests to both cohorts.” 06/18/13
014 It is unclear what information IARPA will use to assess the investigators’ research track record or what weighting this is given in the assessment process. Is part G of Section 4.B.1.3. (“Offeror’s previous accomplishments”) the only part of the proposal, which covers this aspect? In addition to Section 4.B.1.3, part G, please review Section 4.B.1.2, part E, “Project Contributors.” For a description of how offerors’ experience and expertise will be evaluated, please review Section 5.A.4., “Relevant Experience and Expertise”. 06/18/13
015 For Metric 2C, would a genetic predictor be considered an ‘Underlying Neurobiological Mechanism’? As stated on page 7 in BAA Section 1.A.3.iii.,“…offerors must provide a description of … how changes in each mechanism will be analyzed and attributed to the proposed intervention(s).” 06/18/13
016 Given that we are proposing to conduct research outside of the US, would our proposal be considered equally against US based proposals, or would US based protocols be considered preferentially? Please review BAA Section 3.A. on page 23 for full information on eligibility of foreign applicants, which also stipulates compliance with necessary US laws and governing statutes, including regulations for human subject protection. 06/18/13
017 The BAA mentions that “the Government reserves the right to negotiate the type of award instrument (…)’ (cf. p. 22 in the BAA). Under the assumption that our proposal would be selected for funding, could we conduct our proposed research program on a grant basis? As stated in the BAA Section 4.B.2. (page 33), Cost Volume, award instruments requested may include “cost-plus-fixed-fee (CPFF), cost-contract—no fee, cost sharing contract – no fee or other type of procurement contract (specify).” While the Government reserves the right to negotiate the type of award instrument, at this time, grants are not allowable. 06/18/13
018 There appear to be some differences between the Proposers’ Day slides and the SHARP BAA. The SHARP BAA supersedes any information in the SHARP Proposers’ Day slides. 06/18/13
019 Is it true that Volume 1, Section 2, Summary of the proposal (page 27) is limited to only 2 pages? If so, is it acceptable to omit information that is being requested in Section 2, cross reference it and place it in Section 3 only? Volume 1, Section 2 of the proposal is limited to 2 pages, as stated on page 27 of the BAA. Offerors shall meet all of the BAA requirements. It is at the discretion of the offeror to determine how best to convey the information in any given section, while respecting the stated limitations/requirements. 06/18/13
020 If a follow-up retention post-test is part of the experiment for the internal cohort, will a second T&E post-test per participant (for use in retention studies) be made available, in addition to the immediate post-intervention post-test? It is anticipated that no additional post-tests, beyond what is stated in the BAA, will be provided. If an offeror proposes to test retention, it is at their discretion to propose and justify a solution that best meets the conditions defined in BAA Section 1.A.3.ii., 1.A.3.iii, and 1.A.3.v. 06/21/13
021 If the proposers already have a local IRB approval for a current, ongoing study, could pilot testing be conducted under this protocol? Or must all research for this program take place under a new IRB approved protocol? As stated in BAA Section 1.A.5, page 12, all local IRB approvals will require “a review and approval from the SHARP Government Contracting Agent.” As stated in BAA Section 6.B.5, page 42, “No IARPA funding can be used towards human-subject research until ALL approvals are granted.” 06/21/13
022 Can an academic institution act as a prime contractor? Yes. As stated in Section 3.A., page 22 of the BAA, “All responsible sources capable of satisfying the Government's needs may submit a proposal.” Please review BAA Section 3.A.,“Eligible Applicants” and Section 3.B., “US Academic Organizations” for more information. 06/21/13
023 Will we be considered ineligible or noncompliant if we propose a project that lasts 48 months instead of 42 months? Offerors should propose a solution that best meets the conditions defined in BAA Section 1.A.1, “Overview,” and which aligns with the “Program Structure,” as described in Section 1.A.5, and the “Program Timeline,” as described in Section 1.C. 06/21/13
024 Does the answer to question #003 imply that for the T&E cohort, there will be approximately 225 subjects split between the active control and non-active control, and additionally 225 subjects for the intervention, for a total of 450? Or 225 for each of the two controls, for a total of 675? ** Please note this answer was revised on July 1, 2013.

As stated in Section 1.A.5 (page 13) of Amendment 02 of the BAA, offerors should plan to test 110 subjects per group in the T&E cohort. At a minimum, therefore, offerors should plan to test 330 subjects in the T&E cohort to reflect at least one intervention group, an active control, and a non-active control.
06/21/13
025 Is there a Small Business Innovation Research (SBIR) mechanism associated with this BAA? No, there is a not a Small Business Innovation Research (SBIR) mechanism associated with this BAA. 07/08/13
026 Table 1 suggests that metrics that relate specifically to the internal cohort (Metrics 2a and 2b) are to be evaluated only in Month 11 of Phase 1, while Metric 2c, which is applied to both the internal and T&E cohorts, will be evaluated at both Month 11 and Month 22. Does there need to be an internal cohort trained and tested for Phase 1a and a new set of internal cohort subjects run for Phase 1b? There is no requirement for a new set of internal cohort subjects for Phase 1b. It is at the discretion of the offeror to best propose a solution that meets the conditions defined in BAA Section 1.A.5. 07/08/13
027 Is Metric 2c at Month 11 specific to the internal cohort and at Month 22 specific to the T&E cohort? Yes. Per Table 1 and the description of Metric 2c (“Underlying Neurobiological Mechanism(s) in Internal and T&E Cohorts”) in the SHARP BAA Section 1.B.1, pages 16 and 18, respectively, Metric 2c will apply to internal cohort(s) at Month 11, and to T&E cohort(s) at Month 22. 07/08/13
028 Would the government be considered a covered entity under HIPAA (Health Insurance Portability and Accountability Act) for this effort? If so, would a performer be considered a business associate under HIPAA? IARPA is not a covered entity as defined by HIPAA since it is not acting as a health plan, health care provider, and/or health care clearinghouse. IARPA is also not considered a business associate, as defined by HIPAA. Offerors should determine whether their organization(s) is considered a covered entity. Please see the U.S. Department of Health and Human Services website for more information on HIPAA (http://www.hhs.gov/ocr/privacy/index.html).

Please note that, as stated in BAA Section 4.B.1.3. on page 30, “Offerors proposing to obtain new data sets must ensure that their plan for obtaining the data complies with U.S. Laws and where applicable…laws and policies regarding privacy protection of U.S. Persons.”
07/08/13
029 The BAA requires an internal cohort and a T&E cohort. Are additional cohorts allowable? It is at the discretion of the offeror to best propose a solution that meets the goals and conditions defined in BAA Section 1.A.3. 07/08/13
030 Are the 110 T&E subjects listed on page 13 of BAA Amendment 02 the number required across both phases (Phase 1a/1b and 2) or in each phase? That is, is it expected that a total of 220 subjects per phase will be studied if there is a 2 group design? As stated in Section 1.A.5 (page 13) of Amendment 02 of the BAA, offerors should plan to test 110 subjects per group in each Phase’s T&E cohort. At a minimum, therefore, offerors should plan to test 330 subjects in each Phase’s T&E cohort to reflect at least one intervention group, an active control, and a non-active control. 07/08/13
031 If our power calculations suggest we require fewer numbers of T&E subjects than the 110 listed on page 13 of BAA Amendment 02, will that be reviewed unfavorably? As stated in Section 1.A.5 on page 13 of BAA Amendment 02, there are no anticipated restrictions on the number of T&E subjects that can be tested, but at a minimum offerors should plan to test approximately 110 subjects per group (intervention(s) and controls) on the T&E ARP outcome measure. 07/08/13
032 Is it required to run separate studies in each phase, or can the proposed Phase 1 study design and enrollment continue into phase 2, provided milestones at each stage have been met? For example, can the 110 T&E subjects per group in each intervention in Phase 1 be the same subjects in Phase 2? As stated in Section 1.A.5 on page 14 of the BAA amendment, “Offerors should also plan cost estimates for enrolling and testing new internal and T&E cohorts in Phase 2. It is expected, but not required, that subjects participating in Phase 2 should not have previously participated in Phase 1.” This expectation is based, in part, on the requirements specified in Section 1.A.3.ii (page 5), where, in order to “minimize practice effects associated with repeated testing on ARP measures, the T&E cohort must not be exposed to any ARP measures other than those administered via the T&E Battery at pre-test and post-test sessions.” Hence it is at the offeror’s discretion to propose a solution that best meets these restrictions while meeting the goals and conditions specified in Sections 1.A.3.iii. and 1.B.1. 07/17/13
033 Is it compliant and/or desirable to obtain data on neurobiological mechanisms (Metric 2c) during the completion of primary and/or secondary outcome measures (Metric 2a and 2b)? Provided an offeror’s proposed solution meets the goals and conditions defined in BAA Sections 1.A.3.iii. and 1.B.1., it is at the discretion of the offeror as to when and how to obtain data on neurobiological mechanisms (Metric 2c). 07/17/13
034 Would tailoring of interventions based on parameters other than the neurobiological parameters collected be disallowed for Phase 1a and Phase 1b? Provided an offeror’s proposed approach meets the goals and conditions defined in BAA Sections 1.A.3.i., 1.A.3.iii., 1.A.iv., 1.A.5, and 1.B.1., then it is at the discretion of the offeror as to whether, and how, to tailor interventions in Phase 1a and/or Phase 1b. 07/17/13
035 If two or more different intervention groups are proposed for phase 1A, is it allowable to exclude the lesser-performing intervention group(s) when moving into phase 1B? Provided an offeror’s proposed approach meets the goals and conditions defined in BAA Sections 1.A.3.ii. and 1.A.5, then it is at the discretion of the offeror as to whether, and how, to exclude lesser-performing intervention group(s) when moving into Phase 1b. 07/22/13
036 If there is no significant neural or behavioral difference between the active and passive control groups in phase 1A, is it permissible to exclude the passive control group in phase 1B? As stated in BAA Section 1.A.3.v. on page 8, “Unless otherwise strongly justified by the offeror, active and non-active control groups are required in both the internal and T&E cohorts.” 07/22/13
037 As indicated in Section 4.B.1.3.F, are cost estimates intended to be broken out for each deliverable at each of the time points defined in Section 4.B.1.3.E on pages 30-31 of the BAA? Can they be broken out by technical task and/or perhaps month, which can then be tied to deliverables? In accordance with Section 4.B.2. on page 34 the detailed estimated cost breakdown in Volume 2 of the proposal shall include, “Total cost broken down by major task,” which may include numerous deliverables per task. 07/22/13
038 If the proposed experimental protocol includes interim cognitive tests administered during the intervention regime, would this test administration time count towards the weekly intervention time budget of 210 minutes? As defined in BAA Section 1.A.3.i., “Interventions cannot require more than a total of 210 minutes of active participation in any given calendar week.” Please review the definition of “interventions” provided in BAA Section 1.A.3.i. on page 4. Offerors should justify whether any proposed interim cognitive tests fall within or outside of the BAA’s definition of an intervention. 07/22/13
039 Does the causal model deliverable have any additional requirements in addition to: a) relationships between independent and dependent variables; b) testable hypotheses of individual differences; c) testable hypotheses of neurobiological mechanisms? As stated on page 2 of the BAA, evidence-based causal models should “both advance the theoretical bases for the proposed intervention(s) as well as assist in tailoring the proposed intervention(s) for individuals in order to optimize benefits.” Hence, it is at the discretion of the offeror to best propose a solution that meets the requirements for causal models as described in BAA Section 1.A.3.iv., “Causal Model.” 07/22/13
040 Are we required to reach the specified statistically-significant improvement in each of the outcome measures that we are going to use? Or in one/some of them? In the latter case, when should we decide which one(s): after our pilot study in phase 1a? Or by the time of proposal submission? Please review SHARP Program milestones and metrics shown in Table 1 on page 16 and described in BAA Section 1.B.1.

As stated in BAA Section 1.B.1 on page 17, “Offerors will include in their proposal specific metrics IARPA may use to evaluate intervention effectiveness for each of these outcome measures, and which will allow the Government to assess their progress towards reaching Metric 1. The proposal must describe and justify the quantitative metrics for each outcome measure that the Government will be able to review and assess at each milestone and interim waypoints.”
07/22/13
041 Do we need to record neurophysiological measures in all subjects - or can we use a subset of subjects? Please review BAA Sections 1.A.3.iii., page 7, “Recognizing time and resource limitations, performers will not be required to assess these mechanisms in all subjects, but must be able to measure them in a sufficiently powered subset of subjects in each cohort to be able to demonstrate statistically-significant differences between intervention groups and control groups.” 07/22/13