V. APPLICATION REVIEW INFORMATION
1. EVALUATION CRITERIA
Your application will be reviewed and scored according to the quality of your response to the requirements listed below for developing the Project Narrative (Sections A-D). These sections describe what you intend to do with your project.
- In developing the Project Narrative section of your application, use these instructions along with any additional instructions found in the NOFA. These are to be used instead of the “Program Narrative” instructions found in the PHS 5161-1.
- You must use the four sections/headings listed below in developing your Project Narrative. Be sure to place the required information in the correct section, or it will not be considered. Your application will be scored according to how well you address the requirements for each section.
- Reviewers will be looking for evidence of cultural competence in each section of the Project Narrative. Points will be assigned based on how well you address the cultural competence aspects of the evaluation criteria. SAMHSA’s guidelines for cultural competence can be found on the SAMHSA web site at www.samhsa.gov. Click on “Grant Opportunities.”
- The Supporting Documentation you provide in Sections E-H and Appendices 1 through 5 will be considered by reviewers in assessing your response, along with the material in the Project Narrative.
- The number of points after each heading below is the maximum number of points a review committee may assign to that section of your Project Narrative. Bullet statements in each section do not have points assigned to them. They are provided to invite the attention of applicants and reviewers to important areas within each section.
Section A: Statement of Need (15 points)
- Describe the problem the project will address. Describe the national significance of the problem. Documentation of need may come from a variety of qualitative and quantitative sources in the professional literature. The quantitative data could also come from national data available regarding mental health and substance use needs, gaps, and priorities. For example:
- Applications focusing on substance abuse might draw from SAMHSA’s National Household Survey on Drug Use and Health (NHSDUH); Drug Abuse Warning Network (DAWN); and Drug and Alcohol Services Information System (DASIS), which includes the Treatment Episode Data Set (TEDS).
- Applications focusing on mental health might draw on data available from the National Association of State Mental Health Program Directors (NASMHPD), SAMHSA (www.samhsa.gov/cmhs/MentalHealthStatistics), or other sources.
Qualitative sources may also include conclusions of conferences and events of national significance.
- Describe the target population for the practice, including demographic information. Be sure to check the NOFA for any specific requirements regarding the target population and catchment area. Discuss the target population’s language, beliefs, norms and values, as well as socioeconomic factors that must be considered in delivering programs to this population.
- Review the literature that demonstrates a need to develop or adapt an effective practice for the target population. Demonstrate through the literature review that current evidence-based approaches to the problem do not exist or have not been evaluated for the specific target populations, or that approaches of greater clinical or cost effectiveness are needed.
- Demonstrate that the need in the community in which the project will be carried out is of sufficient magnitude that an adequate evaluation of the practice can be conducted. To the extent possible, use locally generated data or State data such as that available through State needs assessments.
- Check the NOFA for any additional requirements.
Section B: Proposed Approach (30 points)
- Describe the practice proposed for evaluation. Document that the practice has been in place and operational for at least one year prior to the application due date.
- Describe how the proposed practice will respond to the needs described in Section A of your Project Narrative and is consistent with the program purpose described in the NOFA. Address any and all program requirements specified in the NOFA.
- Discuss the potential effectiveness of the practice proposed for evaluation. Why has this practice been selected? Present the theoretical underpinnings, core principles, and major assumptions of the proposed practice. Outline the key operational elements of the practice and summarize any relevant literature.
- Identify any necessary collaborators on the project, including their roles and responsibilities. Demonstrate their commitment to the project. Include letters of support in Appendix 1: Letters of Support. Identify any cash or in-kind contributions that will be made to the project by the applicant or other partnering organizations.
- Describe your experience with similar collaborative projects, and explain why you believe you will be able to sustain this collaboration throughout the project period.
- If applying for combined Phase I and II, describe the extent to which the practice has been previously developed, implemented, stabilized, and documented. Include a description of the extent to which the support system needed for full implementation of the proposed practice is in place – e.g., community collaboration and consensus building; alignment of management information systems, policies, and funding mechanisms; documentation of core elements of the practice; reliable recruitment and intake procedures; quality assurance and accountability mechanisms; training and overall readiness of those implementing the practice; and involvement of families and consumers in the project.
- If applying for Phase II only, show that the practice is ready for systematic evaluation by providing documentation, in Appendix 2, that includes all of the following:
- A logic model depicting the principles and concepts underlying the practice.
- A copy of the Title Page and Table of Contents for a manual describing the practice in detail that would allow others to replicate the practice, and details on how the manual can be acquired.
- Documentation of how critical stakeholders were included in the development of the practice.
- A detailed description of the population that the practice is designed to serve, and demographic characteristics of the people served by the practice over the past year.
- Demonstration of stability in the number of people being served by the practice.
- Documentation that staff are trained in the practice (via the number and percentage of staff trained), and a mechanism for ongoing training for any new staff.
- Evidence demonstrating that the practice is in full operation and that a routine service delivery process is in place.
- Pilot outcome results. (Note: Collection of these data need not include an extensive set of outcomes systematically collected on all participants, but quantitative project data should provide some indication that key outcomes are being achieved.)
- Present the goals and measurable objectives of the project. Describe why the practice can better be evaluated for effectiveness following completion of the grant activities. For applications that include Phase I, include in your description how achievement of your goals will fulfill the Performance Expectations cited in Section I-2 of this document.
- Describe the action steps to accomplish the goals and objectives. Demonstrate that the action steps will lead to successful accomplishment of the goals and objectives.
- Describe the potential barriers to successful conduct of the proposed project and how you will overcome them.
- Describe how the proposed project will address issues of age, race/ethnicity, culture, language, sexual orientation, disability, literacy, and gender in the target population.
- Check the NOFA for any additional requirements.
Section C: Evaluation Design and Analysis (40 points)
- Describe in detail your evaluation design for determining the effectiveness of the practice. For applications that include Phase I, describe your process evaluation to determine that the practice is in full operation, as well as how you will track the number and percentage of staff fully trained in the practice.
- Describe the process and outcome evaluation protocols you intend to use. Include in Appendix 3 evaluation instruments to be used. Describe any literature or pilot testing done to verify the validity and reliability of the instruments to be used or how you plan to develop the instruments during the grant period.
- Discuss the reliability and validity of evaluation methods and instrument(s) in terms of the gender/age/culture of the target population.
- Describe how you will develop and manage a database management system to record participant demographic characteristics, practice outcomes, service utilization, practice costs, and satisfaction of stakeholders with the practice.
- Describe how the integrity of the practice will be assessed using a fidelity (see Glossary) scale. If no fidelity scale currently exists for the practice, describe the process by which you will develop one during the grant period. Describe how you will document and assess changes to the model that occur throughout the project.
- Document your ability to collect and report on the required performance measures as specified in the NOFA, including data required by SAMHSA to meet GPRA requirements. Specify and justify any additional measures you plan to use for your grant project.
- Describe how you will analyze the data collected. Include any analyses that will be done to determine the effectiveness of the practice for diverse subgroups, as well as the satisfaction of various stakeholder groups with the practice.
- Describe how your process evaluation will document the role of critical stakeholders in the development and/or evaluation of the practice.
- Check the NOFA for any additional requirements.
Section D: Management Plan and Staffing (15 points)
- Provide a realistic time line for the project (chart or graph) showing key activities, milestones, and responsible staff. [Note: The time line should be part of the Project Narrative. It should not be placed in an appendix.]
- Discuss the capability and experience of the applicant organization and other participating organizations with similar projects and populations, including experience in providing culturally appropriate/competent services.
- Provide a list of staff members who will conduct the project, showing the role of each and their level of effort and qualifications. Include the Project Director and other key personnel, such as evaluators and database management personnel.
- Describe the racial/ethnic characteristics of key staff and indicate if any are members of the target population/community. If the target population is multi-linguistic, indicate if the staffing pattern includes bilingual and bicultural individuals.
- If you plan to include an advisory body in your project, describe its membership, roles and functions, and frequency of meetings.
- Describe the resources available for the proposed project (e.g., facilities, equipment), and provide evidence that resources are adequate for conducting a high-quality evaluation of the identified practice.
- Check the NOFA for any additional requirements.
NOTE: Although thebudget for the proposed project is not a review criterion, the review group will be asked to comment on the appropriateness of the budget after the merits of the application have been considered.
2. REVIEW AND SELECTION PROCESS
SAMHSA applications are peer-reviewed according to the review criteria listed above. For those programs where the individual award is over $100,000, applications must also be reviewed by the appropriate National Advisory Council.
Decisions to fund a grant are based on:
- the strengths and weaknesses of the application as identified by the peer review committee and approved by the appropriate National Advisory Council;
- availability of funds; and
- after applying the aforementioned criteria, the following method for breaking ties: When funds are not available to fund all applications with identical scores, SAMHSA will make award decisions based on the application(s) that received the greatest number of points by peer reviewers on the evaluation criterion in Section V-1 with the highest number of possible points (Evaluation Design and Analysis—40 points). Should a tie still exist, the evaluation criterion with the next highest possible point value will be used, continuing sequentially to the evaluation criterion with the lowest possible point value, should that be necessary to break all ties. If an evaluation criterion to be used for this purpose has the same number of possible points as another evaluation criterion, the criterion listed first in Section V-1 will be used first.
 
 
|
