Use the intervention search tool to find the most up-to-date information on the evidence about helping youth and adults with low incomes succeed in the labor force. Begin by going to Find Interventions that Work, and then use the sorting options to show interventions that are rated 
well-supported or 
supported by domain at the top of the search results.
The Pathways Clearinghouse Quality Review Team (QRT) handles any challenges stakeholders make about a review’s findings, the inclusion of a study within the Pathways Clearinghouse, or other individual judgements the Pathways Clearinghouse team makes. The QRT addresses any issues with reviews that stakeholders raise, so long as they are (1) submitted in writing to PathwaysClearinghouse@abtassoc.com, (2) related to a specific study or well-defined set of studies, and (3) coherently explained (and the inquirer is available to answer any clarifying questions).
When a request is submitted to the QRT, a team member first verifies the request meets the criteria listed above. After this confirmation, the team member examines the study and any related materials, discusses the review with the original study reviewers, and presents a summary of the review and any potential flaws to the QRT. The QRT then determines whether the initial review should be revised, notifies OPRE and the inquirer of its findings and, if necessary, edits any Clearinghouse products to reflect the updated review.
An effectiveness rating is the assessment of the Pathways Clearinghouse, based on the existing evidence from impact studies, of the extent to which a given intervention improves a specific type of outcome. The effectiveness rating is a holistic assessment of whether an intervention is likely to produce favorable results if faithfully replicated with a similar population. An effect size is a standardized measure that allows us to make direct and meaningful comparisons across different outcomes, settings, and interventions. Both the effectiveness rating and the effect size can be compared across studies in the Pathways Clearinghouse. The effect size is the impact found in the available research, and the effectiveness rating is the likelihood that an intervention would produce a similar, favorable effect if implemented again with a similar population and context.
An effectiveness rating assesses whether an intervention is likely to produce favorable results if faithfully replicated with a similar population. Outcome domains with well-supported ratings are those that the evidence suggests are most likely to improve if an intervention were replicated with a similar population. Outcome domains with supported ratings have some evidence that the intervention improves them, but the evidence is less conclusive. Outcome domains that receive a rating of not supported have strong or consistent evidence that the intervention is unlikely to produce favorable effects.
However, because implementation challenges and successes often vary and no two implementations of an intervention are identical, the well-supported and supported ratings do not guarantee success.
The four domains for which the Clearinghouse rates effectiveness are earnings, employment, public benefit receipt, and education and training. Within the first three of these domains, the Pathways Clearinghouse reports on short-term, long-term, and very-long term outcomes [the education and training findings are reported at the longest follow-up, and not segmented into these three time periods]. To make the intervention search results display easier to view and navigate, the effectiveness ratings on the search page represent the highest rating given to the short-term, long-term or very-long term outcomes for that intervention. For example, if an intervention has a supported effectiveness rating in the long-term for earnings, but not in the short-term or very-long term, we will display the supported icon for the earnings domain. Users can click on the individual interventions to see whether the effectiveness ratings apply to short-term, long-term, or very-long term outcomes.
The Pathways Clearinghouse’s ability to report on the effects of an intervention are tied to the existing evidence. Not all interventions have been studied along all the domains on which we report [earnings, employment, public benefit receipt, and education and training]. An intervention may have studies that examined effects in some outcomes but not in others. In other cases, the quality of evidence may vary across outcome domains. If we did not find any studies that rated moderate or high quality that studied the intervention’s effect on outcomes in a given outcome domain, that outcome receive an effectiveness rating of no evidence to assess support. In addition, we may not have all of the data to extract an effect size from the original study (as discussed in the previous FAQ). In those cases, we cannot report on the effects on an intervention for those domains.
This has to do with the difference between how we calculate whether or not an intervention has evidence of being effective on a given outcome domain and how we calculate the size of an intervention’s effects on that outcome domain. In order to receive a supported rating, an intervention must have at least one statistically significant, favorable finding and no statistically significant unfavorable findings in the given outcome domain. Effect sizes, on the other hand, are an average of all findings for a given outcome domain, including those that are not statistically significant.
Take, for example, a study that finds three effects on earnings, one of which is statistically significant, and favorable, and two of which are statistically insignificant, but unfavorable. Because the study identified a statistically significant, favorable effect on earnings and no statistically significant unfavorable findings, the intervention receives an effectiveness rating of supported on earnings. In calculating its overall effect on earnings, however, we average all the findings in this domain, the two unfavorable, but statistically insignificant findings, along with the statistically significant, favorable effect. The average of these three findings might result in an overall negative effect on earnings.
Studies with low quality ratings demonstrate little evidence that findings are attributable, in part or in full, to the intervention examined. A low quality rating suggests that there is a high risk of bias. The Pathways Clearinghouse focuses on providing detailed information for studies rated high or moderate quality because these studies have a lower risk of bias, and these interventions are more likely to have contributed to the reported outcomes. In other words, the Pathways Clearinghouse focuses on well-implemented studies because they provide the most relevant, useful information for practitioners and decision makers.