Frequently Asked Questions

The four domains for which the Clearinghouse rates effectiveness are earnings, employment, public benefit receipt, and education and training. Within the first three of these domains, the Pathways Clearinghouse reports on short-term, long-term, and very-long term outcomes [the education and training findings are reported at the longest follow-up, and not segmented into these three time periods]. To make the intervention search results display easier to view and navigate, the effectiveness ratings on the search page represent the highest rating given to the short-term, long-term or very-long term outcomes for that intervention. For example, if an intervention has a supported effectiveness rating in the long-term for earnings, but not in the short-term or very-long term, we will display the supported icon for the earnings domain. Users can click on the individual interventions to see whether the effectiveness ratings apply to short-term, long-term, or very-long term outcomes.

The Pathways Clearinghouse’s ability to report on the effects of an intervention are tied to the existing evidence. Not all interventions have been studied along all the domains on which we report [earnings, employment, public benefit receipt, and education and training]. An intervention may have studies that examined effects in some outcomes but not in others. In other cases, the quality of evidence may vary across outcome domains. If we did not find any studies that rated moderate or high quality that studied the intervention’s effect on outcomes in a given outcome domain, that outcome receive an effectiveness rating of no evidence to assess support. In addition, we may not have all of the data to extract an effect size from the original study (as discussed in the previous FAQ). In those cases, we cannot report on the effects on an intervention for those domains.

Very long-term outcomes are those measured 5 years or more after participants in the study’s intervention group are first offered services.

The Pathways Clearinghouse reports on the effects for study findings that are rated high or moderate quality in our four domains [earnings, employment, public benefit receipt, and education and training]. When the quality of a study is high, that means we can be fairly confident in the study findings because the study finding is solely attributable to the intervention examined. This rating is reserved for study findings from high quality RCTs with low attrition of sample members. The Pathways Clearinghouse also reports effects for moderate quality studies, where we can be somewhat confident in the study findings.

There are several reasons that some findings in a study may not be rated. Specifically:

• Studies may have outcomes that are rated low quality, where we cannot have much confidence in the study findings. Other important factors could have influenced the study findings, and the study did not account for them. The Pathways Clearinghouse does not report outcomes that are rated low quality.
• If the original study provides findings for multiple outcome measures in a given domain, the Pathways Clearinghouse prioritizes findings to review and report based on the outcome measure, following the prioritization process summarized in Exhibit III.2 of the Protocol for the Pathways to Work Clearinghouse: Methods and Standards. For example, if study authors used both surveys and administrative records to assess earnings, Pathways Clearinghouse reviewers select two sets of earnings findings for review: one measured using survey data and one measured using administrative data.

In addition, the Pathways Clearinghouse may not have sufficient information to report on the magnitude of study findings. We calculate an effect size using study-specific data, if we can obtain from the study authors the information needed to do so.