Below we provide responses to frequently asked questions (FAQs). We first present responses to general FAQs. The remaining FAQs are organized according to the review process steps detailed in the Handbook of Standards and Procedures, Version 1.0.
* We are excited to announce the release of the Title IV-E Prevention Services Clearinghouse Handbook of Standards and Procedures, Version 2.0. In an effort to continue to review and rate programs and services as quickly as possible - reviews under the Handbook of Standards and Procedures, Version 1.0 will be ongoing during a transition period. This page will be updated to include FAQs related to the Handbook of Standards and Procedures, Version 2.0 when the Clearinghouse begins reviewing programs and services using this revised version.
A downloadable copy of the second version of the handbook can be found on the Handbook of Standards and Procedures, Version 2.0 page.
A downloadable copy of the original version of the handbook can be found on the Handbook of Standards and Procedures, Version 1.0 page.
The Prevention Services Clearinghouse website includes the working list of programs and services that are currently under review. This can be found on the Programs & Services Planned For Review page.
It is difficult to predict the timeline for review of an individual program and service. Review timelines are influenced by a number of factors including, but not limited to:
- The number of eligible studies examining a particular program or service;
- Whether eligible studies have all the information necessary to complete the review or whether author queries will be needed to obtain additional information;
- If author queries are needed, the timeline for authors to respond to the queries and the information they provide; and,
- Whether any questions come up during the course of the review of studies that require internal or external expert consultation.
For more information, see the fact sheet The Review Process: Navigating Timeline Challenges.
Our goal is to review and rate as many programs and services as quickly as possible to support states’ efforts to improve outcomes for children and families through implementation of the Family First Prevention Services Act. We encourage individuals to sign up for email updates on the Clearinghouse website to receive notifications about changes to the working list of programs and services planned for systematic review and the release of new program or service ratings.
The Handbook of Standards and Procedures, Version 2.0 is now available. To download a copy, visit our Handbook of Standards and Procedures, Version 2.0 page. Consistent with the practice of other federal evidence reviews, standards and procedures may be revised over time as research methods evolve, the needs of the field change, and lessons are learned during the review process. Potential revisions will be informed by a transparent process and build from the existing knowledge of States, federal agencies, researchers, evaluators, program and service developers, experts, and the general public. Opportunities for public feedback through Federal Register Notices will be announced via our email list. To receive updates, please join our email list.
The Clearinghouse has developed a working list of the next programs and services planned for systematic review. To learn more about this list, please visit the Programs & Services Planned For Review page.
The Prevention Services Clearinghouse continues to select programs and services for review using an inclusive process. The Prevention Services Clearinghouse staff have logged all program and service recommendations received since 2018. Programs and services may be identified from the following sources:
- A public call for recommendations on at least an annual basis. The latest public call occurred in the fall of 2023 and is now closed.
- Recommendations received via email to PreventionServices@abtglobal.com; and
- An environmental scan and an inventory of the literature.
Given the high volume of recommendations, the Clearinghouse must prioritize programs and services for review. Particular consideration is given to programs and services recommended by State or local government administrators; rated by other clearinghouses (such as CEBC or HomVEE); recommended by federal partners; and/or evaluated as part of any grants supported by the Children’s Bureau (such as the Title IV-E Child Welfare Demonstrations or Regional Partnership Grants).
To learn more about how we currently prioritize and select programs and services for review, please see Chapter 2 of the Clearinghouse’s Handbook of Standards and Procedures, Version 1.0.
The most recent public call occurred in the fall of 2023 and is now closed. However, all recommendations sent to the Clearinghouse are logged and considered, not only those received in response to the annual public calls. Recommendations may be sent to PreventionServices@abtglobal.com. Submissions may include publicly available literature in support of recommended programs and services.
For more information on what to include in a program or service recommendation, see the next question “What information is helpful to include when I recommend a program or service?”
Recommendations should be sent to PreventionServices@abtglobal.com. All submitters will receive acknowledgment of their submission's receipt, but the Prevention Services Clearinghouse will not inform individual submitters of the outcome of their submissions.
The following information is helpful to include in a cover letter or email:
- Name of program or service: Some programs or services have changed names or are known by multiple names. Please list all known names for the program or service and if applicable, specify the version or adaptation recommended for review.
- Program or service contact information: If available, also include contact information for the program developer, which may include the official program website, the organizational email address and contact phone number.
- Program or service area(s) addressed: Specify: [1] mental health prevention and treatment programs or services, [2] substance abuse prevention and treatment programs or services, [3] in-home parent skill-based programs or services, and/or [4] kinship navigator programs.
- Citation(s) for available book/manual/writings: Programs and services reviewed by the Clearinghouse must be clearly defined and replicable. Programs and services must have available written protocols, manuals, or other documentation that describe how to implement or administer the practice. Please provide a citation with the title, author, date, and publisher for such materials.
- Indicate how the program or service manual(s) can be accessed. If the manual is available to the public (for free or for purchase), please provide a link to the website where individuals may download, request, or purchase the manual. The purpose of this request is to ensure that Clearinghouse staff understand the program as it is designed to be delivered. Any materials included with a nomination will not be made public or re-distributed.
- Note: The Prevention Services Clearinghouse prioritizes programs or services that are in active use. This means that they must be currently available or delivered with a book, manual, or other documentation available in English. Programs and services that are no longer actively used, are defunct or discontinued, or are otherwise not currently practiced or delivered would not meet this criterion.
- How the recommended program or service is relevant to advancing equity and supporting underserved communities (if applicable). Specify how the recommendation supports the goals and requirements of the Executive Order on Advancing Racial Equity and Support for Underserved Communities. For example, specify whether review of the recommended program or service is of particular interest due to the specific population it serves. This may include programs and services that are designed for or have been adapted for specific cultural, ethnic, or racial groups; programs that have been tested with special populations even if they weren’t culturally tailored or designed for them; or programs that aim to serve other populations that have been historically marginalized and/or have historic or ongoing disproportionate representation in the child welfare system.
Submissions may also include electronic version(s) of eligible studies: The Prevention Services Clearinghouse conducts a comprehensive literature search for each program or service prioritized for review, but welcomes the submission of study reports, articles, or other supporting documentation (e.g., lists of published research). Any materials included with a nomination will not be made public or re-distributed.
For more information about the Prevention Services Clearinghouse study eligibility criteria, download the Prevention Services Clearinghouse Handbook of Standards and Procedures, Version 1.0.
Our goal is to review and rate as many programs and services as quickly as possible to support states’ efforts to improve outcomes for children and families through implementation of the Family First Prevention Services Act (FFPSA).
The Prevention Services Clearinghouse will continue to select additional programs and services for review on a rolling basis. Join our email list to be notified of updates!
The Prevention Services Clearinghouse has received a high volume of program and service recommendations to date. This includes hundreds of emails in response to each of the public calls that solicited recommendations for programs and services. Many of these recommendations identify multiple programs and services. The Prevention Services Clearinghouse staff log and review all program and service recommendations and respond to submitters noting that their recommendations have been received. Given the high volume of recommendations, it is not currently feasible within resource and time constraints for staff to send individualized responses to submitters indicating why a program or service recommended has not yet been selected for review. The Clearinghouse’s systematic prioritization process is outlined in the Handbook of Standards and Procedures, Version 1.0 (Section 2.2). The Prevention Services Clearinghouse is working to review and rate as many services and programs as quickly as possible.
To be eligible for review, programs and services must:
- Meet eligibility criteria for at least one of the four program or service areas: (1) mental health prevention and treatment programs or services, (2) substance abuse prevention and treatment programs or services, (3) in-home parent skill-based programs or services, and/or (4) kinship navigator programs.
- Have a book/manual/writings available: Programs and services must be clearly defined and replicable. To meet this criterion, they must have available written protocols, manuals, or other documentation that describes how to implement or administer the practice. Protocols, manuals, or other documentation must be available to the public to download, request, or purchase.
To learn about the Clearinghouse’s standards and procedures, including further detail about program or service eligibility, please visit our website or download the Prevention Services Clearinghouse Handbook of Standards and Procedures, Version 1.0. For a list of programs and services that have been reviewed by the Prevention Services Clearinghouse, visit https://preventionservices.acf.hhs.gov/program.
The Clearinghouse may contact program or service developers for information about available written protocols, manuals, or other documentation that describe how to implement or administer the practice to verify its eligibility. Developers may also be contacted to determine if there are other versions or adaptations of the program or service. Finally, the Clearinghouse may contact developers to review the description of the program or service that will appear on the website.
Developers can support the review process by: clearly listing contact information on the program or service website; promptly responding to developer queries; and reviewing the Prevention Services Clearinghouse’s description of the program in a timely manner.
As described in the Clearinghouse’s Handbook of Standards and Procedures, Version 1.0, programs and services are identified for review using an inclusive process (see Chapter 1). Once identified, the Clearinghouse then selects and prioritizes programs and services for review (see Chapter 2). The Clearinghouse maintains a working list of the next programs and services planned for systematic review. This can be found on the Programs & Services Planned for Review page on the website. Candidate programs and services on this list are evaluated against program or service eligibility criteria, as described in Chapter 2 of the Handbook of Standards and Procedures, Version 1.0. Programs and services added to the working list may later be deemed ineligible for review if they do not meet these eligibility criteria. Reasons a program or service might not meet these eligibility criteria are listed in the Handbook of Standards and Procedures, Version 1.0 under Program Eligibility Criterion 2.1.1 and 2.1.2. These include: 1) a written protocol, manual, or other documentation that describes how to implement or administer the practice is not available to the public at this time or 2) the program or service does not meet the criteria for any of the four program or service areas eligible for inclusion. When this happens, the program appears on the Programs and Services Reviewed page with a description of “This program or service is not eligible for review at this time.”
As noted in Chapter 3 of the Handbook of Standards and Procedures, Version 1.0, the Prevention Services Clearinghouse begins the search by identifying citations from other evidence clearinghouses. A number of evidence clearinghouses overlap in content with the Prevention Services Clearinghouse (see Exhibit 3.1). The list of clearinghouses may change and additional clearinghouses may be used, depending on the program or service selected. We currently review the following clearinghouses:
Blueprints for Healthy Youth Development
The California Evidence-Based Clearinghouse for Child Welfare
The Campbell Collaboration
Cochrane Reviews
CrimeSolutions
Healthy Native Youth
Home Visiting Evidence of Effectiveness
National Registry of Evidence-based Programs and Practices (via Results First Clearinghouse)
Social Programs That Work
Teen Pregnancy Prevention Evidence Review
Washington State Institute of Public Policy
To be eligible for review, a study must:
- Evaluate the version of the program or service currently under review.
- Use a randomized or quasi-experimental group design with at least one intervention condition and a “no or minimal intervention” or “treatment as usual” comparison condition
- Measure and report program or service impacts on at least one eligible outcome
- Be published or prepared in or after 1990
- Be publicly available and published in peer-reviewed journals or in reports prepared or commissioned by federal, state, or local government agencies or departments, research institutes, research firms, foundations or other funding entities, or other similar organizations
- Be available in English
For more information about study eligibility criteria, see Section 4.1 of the Handbook of Standards and Procedures, Version 1.0.
There is no minimum sample size for studies to be eligible for review by the Prevention Services Clearinghouse. Section 4.1 of the Handbook of Standards and Procedures, Version 1.0 describes the six study eligibility criteria, none of which include sample size restrictions.
In the “Individual Study Findings” sections of the respective program and service pages, the Clearinghouse reports sample sizes for each study by contrast in the “N of Participants” column for all contrasts that meet moderate or high evidence standards. Examples of studies with favorable contrasts that meet the moderate or high evidence standards include:
Studies may be ineligible for a variety of reasons, and some studies are ineligible for multiple reasons. The most common reasons that studies are ineligible are listed below:
- They do not examine the version of the program or service currently under review. To be eligible for review, studies of a program or service must all represent similar implementations of the program under review; that is, programs or services may not be substantially modified or adapted from the written protocol, book, manual, or other documentation (manual) or version of the program or service selected for review. Minor modifications to programs or services that are not considered formal adaptations are addressed in Section 4.1.6 of the Handbook of Standards and Procedures, Version 1.0.
- If a study indicates that content was adapted, the Clearinghouse may determine that the study is not evaluating the program or service that was selected for review.
- If a study implements an older version of a program or service selected for review and the older version of the manual is substantially different from the manual selected for review, the Clearinghouse may determine that the study is not eligible for review.
- The study does not use a design that establishes that the program or service, and not other factors, is responsible for the outcomes observed.
- Eligible research designs include randomized or quasi-experimental group designs with at least one intervention condition and at least one comparison condition. Not all studies are designed to support causal inferences about a program or service. For example, studies that use a pre-post design in which all study participants received the intervention cannot isolate the effect of the program from other factors that may explain changes in outcomes, such as children maturing or symptoms naturally improving over time. In such cases, the Clearinghouse cannot be confident that the outcomes reflect the causal impact of the program.
For more information, Section 4.1 of the Handbook of Standards and Procedures, Version 1.0 provides a complete list of study eligibility criteria. The Reporting Guide for Study Authors offers advice to help study authors describe their studies completely and consistently to help facilitate the Prevention Services Clearinghouse’s eligibility review process.
When an individual study is listed in the Studies Reviewed section of a program page as having a rating of Low or Not Eligible for Review, the study citation is followed by the reason for this rating in bold-italic. These reasons include:
- This study received a low rating because it did not meet design confound standards. These standards are described in Section 5.9.3 of the Handbook of Standards and Procedures, Version 1.0.
- This study received a low rating because none of the target outcomes met measurement standards. These standards are described in Section 5.9.2 of the Handbook of Standards and Procedures, Version 1.0.
- This study received a low rating because baseline equivalence of the intervention and comparison groups was necessary and not demonstrated. These standards are described in Sections 5.7 and 5.8 of the Handbook of Standards and Procedures, Version 1.0.
- This study received a low rating because the standards for addressing missing data were not met. These standards are described in Section 5.9.4 of the Handbook of Standards and Procedures, Version 1.0.
- This study received a low rating because it did not meet the statistical model standards. These standards are described in Section 5.9.1 of the Handbook of Standards and Procedures, Version 1.0.
Provide information needed in reporting. As authors write up study results, they can proactively include information required for the review process. Comprehensive reporting up front can prevent the need for author queries. For more information, see the Reporting Guide for Study Authors.
Responding promptly to author queries, when applicable. The Clearinghouse contacts study authors if additional information is needed to assess reliability of measures, attrition, baseline equivalence of treatment and comparison group, or other factors that can influence study ratings. If an author does not respond to a query, reviewers complete the review with the information available. Prompt responses to author queries can help ensure a timely review.
If a program or service has 15 or fewer eligible studies, all studies are reviewed using the design and execution standards described and assessed for risk of harm.
If a program or service has more than 15 eligible studies, all eligible studies are assessed for risk of harm. Study review prioritization criteria are used to determine the order of eligible studies reviewed using the design and execution standards. Once ordered, the first 15 eligible studies will be reviewed using the design and execution standards. If, after review of 15 eligible studies, a program or service has not achieved a rating of well-supported, additional studies will continue to be reviewed in order until the program or service has achieved a rating of well-supported or all eligible studies have been reviewed.
Many programs and services have been adapted (e.g., modified to address particular issues or populations). The Prevention Services Clearinghouse reviews each adaptation to determine if the program or service has been substantially modified or adapted from the version that has been selected for review. The Prevention Services Clearinghouse reviews eligible studies only of the version selected for review.
Exhibit 4.1. within the Handbook of Standards and Procedures, Version 1.0 (shown below) provides examples of ‘eligible’ adaptations (i.e., adaptations that have not substantially modified or adapted the program or service from the one selected for review). For example, if a program modestly changes the session frequency or duration, this may be reviewed as part of the version that was selected for review. Exhibit 4.1 below also provides examples of adaptations that would be considered substantial and result in a different version of the program or service than the one selected for review. For example, substantial changes to a program or service’s enrollment or eligibility criteria.
When there are multiple versions of a program or service, the Prevention Services Clearinghouse may select just one version as indicated by a specific book/manual/other documentation to review at a particular time. Other versions of the program or service may be reviewed by the Prevention Services Clearinghouse in the same round of review or in later rounds of review.
For additional information see Section 4.1.6 of the Handbook of Standards and Procedures, Version 1.0.
As described in Section 4.1.4 of the Handbook of Standards and Procedures, Version 1.0, eligible studies must use a randomized or quasi-experimental group design with at least one intervention condition and at least one comparison condition. Intervention and comparison conditions may be formed through either randomized or non-randomized procedures and the unit of assignment to conditions may be either individuals or groups of individuals (e.g., families, providers, centers). Comparison groups may be formed using a variety of methodologies, including, for example, both propensity score matching and weighting strategies. Designs in which the same group of individuals is evaluated before and after receiving an intervention are not eligible (i.e., single group pretest posttest designs). This includes interrupted time series designs without comparison groups. The Prevention Services Clearinghouse does not currently review studies that use regression discontinuity designs.
Studies must measure and report program or service impacts on at least one eligible target outcome, as described in Section 4.1.5 of the Handbook of Standards and Procedures, Version 1.0. Programs and services in the areas of mental health, substance abuse, and in-home parent skills must target outcomes in the domains of (a) Child Safety, (b) Child Permanency, (c), Child Well-Being, and/or (d) Adult Well-Being. Programs and services in the area of kinship navigator must target outcomes in the domains of (a) Child Safety, (b) Child Permanency, (c) Child Well-Being, (d) Adult Well-Being, (e) Access to Services, (f) Referral to Services, and/or (g) Satisfaction with Programs and Services. Eligible target outcomes are defined as follows:
The Prevention Services Clearinghouse reviews the following domains of Child Safety:
- Child Welfare Administrative Reports. Substantiated or unsubstantiated child maltreatment from administrative records. Eligible indicators include, but are not limited to, substantiated and unsubstantiated reports of abuse or neglect, investigations of abuse and neglect from administrative records, and recurrence of abuse and neglect from administrative records.
- Self-Reports of Maltreatment. Eligible indicators include victim and perpetrator reports of abuse or neglect and questionnaire or interview instruments that directly assess abusive behavior or neglect.
- Maltreatment Risk Assessment. Eligible indicators include child maltreatment risk assessments.
- Medical Indicators of Maltreatment Risk. Eligible indicators include administrative, questionnaire, or interview instruments assessing childhood injuries, ingestions, emergency room visits, hospitalizations and any other indicators of childhood injuries.
The Prevention Services Clearinghouse reviews the following domains of Child Permanency:
- Out-of-Home Placement. Any situation where a child is removed from the family home. Eligible indicators include, but are not limited to, any out-of-home placement, placement to foster care, reports of the caregiver relinquishing her or his role, and time to placement in out-of-home care.
- Least Restrictive Placement. Included in this subdomain are measures that assess the restrictiveness or disruptiveness of out-of-home placement. This subdomain focuses on improving the environments/settings into which children are placed, including favoring kinship placements over non-kin or institutional placements or placements that maintain connections to the child’s community versus those that do not. Outcomes must be operationalized with more than two placement settings, as binary measures for which the reference category is another out-of-home placement setting, or as movement from more restrictive/disruptive to less restrictive/disruptive settings. Eligible indicators include, but are not limited to, hierarchies of least restrictive preference (e.g., kin placement, family foster care, therapeutic care, group home, residential, hospitalization, and incarceration).
- Placement Stability. Placement stability refers to the stability of out-of-home placement (e.g., that children are in placements that are disrupted infrequently). This subdomain focuses on the number of placement disruptions (planned and unplanned) or number of out-of-home placements. Eligible indicators include, but are not limited to, number of placement changes or disruptions of placements, and re-entries or failed exits/reunifications or adoptions.
- Planned Permanent Exits. Planned permanent exits from out-of-home care refer to placements or time to placement to a more permanent status, including reunification, guardianship, and adoption. Eligible indicators include, but are not limited to, measures of the amount of time to reunification, guardianship, or adoption and reunification rates.
The Prevention Services Clearinghouse reviews the following domains of child well-being, the specific nature of which may vary with age:
- Behavioral and Emotional Functioning. Characteristics and behaviors relating to the ability to realize one’s potential, cope with daily activities, and work and play productively and fruitfully. Both strengths-based and deficit-based indicators are eligible. Examples include measures of externalizing behaviors (e.g., aggressive behavior, disruptiveness, impulsive behavior), internalizing behaviors (e.g., depression, anxiety, mood or thought problems), mental/behavioral health diagnoses, positive behavior, resilience, self-regulation or self-control, and emotional adjustment.
- Social Functioning. Skills and capabilities relating to the ability to develop, maintain, and manage interpersonal relationships (e.g., social skills, assertiveness, cooperation, empathy, social adjustment, peer relations, rebelliousness, defiance, and other similar characteristics related to interpersonal interactions and relationships).
- Cognitive Functions and Abilities. Abilities related to reasoning, knowledge, problem-solving, mental processing, executive functioning, and the like. Eligible measures include intelligence tests, developmental assessments, measures of visual or spatial processing, and other indicators of cognitive functions and abilities.
- Educational Achievement and Attainment. Educational achievement refers to the extent to which students master academic content. Eligible measures include composite or subject-specific (e.g., reading, mathematics) standardized achievement test scores or overall grade point averages. Educational attainment refers to student progress through school or the completion of a degree, certificate, or program. Eligible measures of attainment include grade promotion, high school graduation or dropout rates, certificate or degree completion rates, and other indicators for educational attainment.
- Physical Development and Health. Characteristics related to the healthy functioning of the body may include indicators of physical health (e.g., Body Mass Index), physical capabilities (e.g., motor skills), normative indicators of healthy development (e.g., height), and any other measure relating to healthy (or unhealthy) physical development.
- Substance Use or Misuse. Measures of substance use or misuse may involve any substances and may be self- or other-reported, clinical tests such as urinalysis, or any other measure that provides an assessment of the participants’ substance use behavior. Measures must describe actual use or misuse, such as frequency or quantity of use, type of use, use/no use, time since last use, etc. Substance use diagnoses (e.g., from a clinical interview or DSM criteria) are considered eligible outcomes in this domain. Measures that do not directly index substance use or misuse (e.g., drug-related criminal or delinquency activity such as selling drugs, drug knowledge, behavioral intentions to use or not, etc.) are not eligible in this domain, but may meet the requirements for other outcome domains.
- Delinquent Behavior. Delinquent behavior refers to behavior chargeable under applicable laws, whether or not apprehension occurs or charges are brought. Chargeable offenses also include “status” offenses (e.g., runaway, truancy, curfew violations).
The Prevention Services Clearinghouse reviews the following domains of adult well-being:
- Parenting Practices. Parenting practices include a range of practices and behaviors focused on developing strong, functional relations between parents or caregivers and children and the parents or caregivers’ abilities to successfully manage child socialization and support child development, health, and well-being in an effective and constructive manner. Measures may include items about basic elements of caregiving, such as feeding and physical care; communication and listening; nurturing, loving, or supportive behavior; rules and consequences; setting boundaries; warmth; scaffolding children’s behavior to develop self-discipline; parent-child relationships, and the like. Measures may index either positive parenting practices or negative parenting practices.
- Parent/Caregiver Mental or Emotional Health. Mental or emotional health refers to a parent’s/caregiver’s ability to cope with daily activities, realize his or her potential, and interact productively in the world. Both strengths-based and deficit-based indicators are eligible. Examples include measures of externalizing behaviors (e.g., aggressive behavior), internalizing behaviors (e.g., depression, anxiety, mood or thought problems), mental/behavioral health diagnoses, parent/caregiver stress, relationship stress, positive behavior, resilience, and emotional adjustment.
- Parent/Caregiver Substance Use or Misuse. Measures of substance use or misuse may involve any substances and may be self- or other-reported, clinical tests such as urinalysis, or any other measure that provides an assessment of the participants’ substance use or misuse. Measures must describe actual use or misuse, such as frequency or quantity of use, type of use or misuse, use/no use, time since last use, etc. Substance use diagnoses (e.g., from a clinical interview or DSM criteria) are considered eligible in this domain. Measures that do not directly index substance use or misuse (e.g., drug-related criminal or delinquency activity such as selling drugs, drug knowledge, behavioral intentions, etc.) are not eligible in this domain, but may meet the requirements for other outcome domains.
- Parent/Caregiver Criminal Behavior. Criminal behavior refers to behavior chargeable under applicable laws, whether or not apprehension occurs or charges are brought.
- Family Functioning. Family functioning refers to the capacity or lack of capacity of a family to meet the needs of its members and includes physical care and maintenance of family members; socialization and education of children; and, economic and financial support of the family.
- Physical Health. Refers to the physical health of parents or caregivers and can include a variety of indicators including blood pressure; weight, obesity, or body mass index (BMI); chronic conditions such as asthma or diabetes; and, healthy lifestyle behaviors such as diet and exercise.
- Economic and Housing Stability. Economic and housing stability includes indicators of financial or economic stability (e.g., level of income, employment/unemployment, financial assistance) and/or housing stability (e.g., number of moves, quality of housing, homelessness).
Additional outcomes that are eligible for kinship navigator programs include:
- Access to Services. Access to services refers to a parent, caregiver, or family’s knowledge of and ability to access, or utilization of services to support the family’s financial, legal, social, educational, and/or health needs such as medical care, financial assistance, and social services. Parent/caregiver self-reports, informed collateral reports (e.g., from therapists or case managers), or administrative records are eligible indicators for Prevention Services Clearinghouse reviews.
- Referral to Services. Referral to services may include referrals to any needed financial, legal, social, educational, or health services. Measures may be obtained from parent/caregiver self-reports, therapist or provider reports or records, or administrative records. Examples include the presence or absence of referrals or counts/frequencies of referrals.
- Satisfaction with Programs and Services. Satisfaction with programs and services refers to parent or caregiver satisfaction with the programs and services to which they are referred or which they receive as part of a kinship navigator program.
Section 4.1.4. of the Handbook of Standards and Procedures, Version 1.0 indicates that eligible comparison groups must be “no or minimal intervention” or “treatment as usual” groups.
“No or minimal intervention” comparison group members may receive handouts, referrals to available services, or similar nominal interventions. Such additional services must be brief (i.e., one session or less) and/or predominantly information-based (e.g., pamphlets about child development, psychoeducation about a specific disorder).
“Treatment as usual” comparison group members may be receiving services in their communities already or they may be offered services as part of the research study. Services must be clearly described as the usual or typical services available for that population in the study. The Prevention Services Clearinghouse considers a comparison groups to be “treatment as usual” under either of the following two conditions.
- Condition 1: Usual or typical services. Comparison groups that are already receiving services in their community or are offered services that they would have received in the absence of the study meet this condition. These comparison groups do not receive services because of the study (i.e., they do not receive anything they wouldn’t already receive in the absence of the study). However, it is acceptable for studies with comparison groups in this category to provide or offer minimal intervention (as defined above) along with the treatment as usual. To be eligible under Condition 1, the study or supporting documentation must provide specific information to demonstrate that only usual or typical services were available and that any services offered by the study in addition to usual or typical services were minimal.
- Condition 2: Services consistent with usual or typical services. Comparison groups that are offered services as part of the study that are clearly described in the study as consistent with the usual or typical services that would be received by individuals or families similar to those in the study may also be eligible. In such cases, the study or supporting documentation must provide specific information to justify that the services offered to the comparison group participants are consistent with what individuals or families like those in the study could be expected to receive in the absence of a study.
In studies with multiple eligible comparison groups, reviewers select one eligible comparison per eligible outcome at each time point instead of comparing the same intervention group to multiple comparison groups for all outcomes and time points. Selection of an eligible comparison group for each outcome at each time point is based on the group that receives the least intensive services in order to maximize the treatment contrast.
Studies in which the program or service under review is delivered in combination (or bundled) with another program or service may exhibit a design confound. In such cases, it is impossible to separate how much of the observed effect was related to the intervention and how much to the confounding factor. Thus, the study cannot meet evidence standards and will receive a low rating (see Section 5.9.3. of the Handbook of Standards and Procedures, Version 1.0 for additional information on design confounds). We illustrate three scenarios of relevance to bundled interventions below:
- Intervention + Treatment as Usual vs. Treatment as Usual Only. Studies of bundled intervention conditions do not exhibit a design confound when the program or service under review is bundled with treatment as usual and the comparison group also receives treatment as usual.
- Intervention + Treatment as Usual vs. No/Minimal Treatment. Studies of bundled intervention conditions may exhibit a design confound when the program or service under review is bundled with treatment as usual but the comparison group does not receive treatment as usual. Studies with this configuration of conditions would only be determined to be free of a design confound if the treatment as usual services bundled with the program or service under review are weak, minimal, or unlikely to affect study outcomes.
- Intervention + Second Intervention vs. No/Minimal Treatment or Treatment as Usual. Studies of bundled intervention conditions in which the second intervention in the bundle is not treatment as usual (i.e., the intervention condition receives the program or service under review and another program or service) are ineligible for review unless both programs or services are selected for review as a bundle. If a bundle of services is selected for review, the eligibility criteria for comparison groups apply; that is, the comparison groups must be no treatment or minimal treatment, or treatment as usual groups.
The Prevention Services Clearinghouse uses unique standards and procedures developed in accordance with the statutory requirements detailed in the Family First Prevention Services Act (FFPSA) of 2018. For example, though the California Evidence-Based Clearinghouse for Child Welfare (CEBC) uses a peer-review process to review and rate studies of programs and services, the Prevention Services Clearinghouse is required to conduct an independent, systematic evidence review. In addition, different from CEBC, the Title IV-E Prevention Services Clearinghouse allows programs or services with rigorous quasi-experimental evidence to achieve a well-supported or supported rating. The Prevention Services Clearinghouse also includes review of publicly available research studies including those that do not appear in peer-reviewed journals.
Similarly, different from Home Visiting Evidence of Effectiveness (HomVEE) review, the rating criteria and outcome domains identified in FFPSA do not directly align with the criteria established by HHS for evidence-based home visiting models. For example, in accordance with the statutory requirement of FFPSA, the Prevention Services Clearinghouse criteria for sustained effects are based on length of time after the end of treatment, whereas HomVEE criteria for sustained effects are based on length of time after enrollment.
Can a program or service be re-reviewed using the Handbook of Standards and Procedures, Version 2.0?
Yes. As noted within Section 8.5.1 of the Handbook of Standards and Procedures, Version 2.0, programs and services reviewed by the Prevention Services Clearinghouse under a prior version of the Handbook of Standards and Procedures may be considered for re-review under the current version of the Handbook of Standards and Procedures if a re-review has the potential to change the program or service rating. Requests for the re-review of a program or service rating should be submitted by emailing PreventionServices@abtglobal.com. To help facilitate consideration of the re-review, requests should specify the rationale for the request and provide any supporting documentation. Examples of supporting documentation could include citations for new research studies not previously reviewed or documentation of potential errors or missing information identified.
Yes. As noted within Section 7.4.1 of the Handbook of Standards and Procedures, Version 1.0, programs and services reviewed for the Prevention Services Clearinghouse may be considered for re-review due to missing information or errors in the original review, or due to the emergence of substantial new evidence that has the potential to change program or service ratings. Prevention Services Clearinghouse staff keep track of the dates that programs and services are reviewed and periodically assess the extent of new evidence available. Periodically, content experts may be consulted to determine if new research is available and if the available research has the potential to change the rating of the program or service. Requests for the re-review of a program or service rating should be submitted by emailing PreventionServices@abtglobal.com. To help facilitate consideration of the re-review, requests should specify the rationale for the request and provide any supporting documentation. Examples of supporting documentation could include citations for new research studies not previously reviewed or documentation of potential errors or missing information identified.
As noted within Section 7.4.2 of the Handbook of Standards and Procedures, Version 1.0, individual studies reviewed by the Prevention Services Clearinghouse may be considered for re-review due to missing information or errors in the original review. If errors or missing information are identified, the Prevention Services Clearinghouse follows standard procedures for re-review. This includes assigning different, blinded reviewers to conduct any re-reviews. If the re-review determines the original review to be in error, the error is corrected on the website. All correspondence regarding re-reviews is logged and maintained by Prevention Services Clearinghouse staff. Requests for the re-review of an individual study should be submitted by emailing PreventionServices@abtglobal.com. To help facilitate the re-review, specify the rationale for the request and provide any supporting documentation.
When an individual study is listed in the Studies Reviewed section of a program page as having a rating of Low or Not Eligible for Review, the study citation is followed by the reason for this rating in bold-italic. For more information on these reasons, see the questions “How can I tell the reason why a study received a low rating?” and “What are common reasons studies are not eligible for review?” found under the FAQ category of Study Eligibility Screening and Prioritization above.
For more information on why studies do not meet the Clearinghouse’s design and execution standards, or receive low ratings for design and execution, see Chapter 5 of the Handbook of Standards and Procedures, Version 1.0 and the OPRE fact sheet, How Does the Prevention Services Clearinghouse Rate the Design and Execution of Studies?
