Frequently asked questions

The Pathways to Work Evidence Clearinghouse’s ability to report on the effects of a program depend on the existing evidence. If the program did not have any study findings rated moderate or high for a given outcome, that outcome receives an effectiveness rating of cannot assess support. There are several reasons why study findings may not receive a moderate or high rating. Some studies may not report on all Pathways to Work outcomes (earnings, employment, public benefit receipt, and education and training). A program may have studies that examine effects for some outcomes but not others. In other cases, the quality of evidence may vary across outcomes. In addition, we may not have all the data needed to calculate an effect size from the original study. In these cases, we cannot report the effects of a program for those outcomes.

Very long-term outcomes are those measured 5 years or more after participants in the study’s program group are first offered services. To view very long-term outcomes for a program, click on the program’s page and navigate to the Effectiveness rating and effect by outcome domain table.

Pathways to Work reports on the effects for study findings that are rated high or moderate quality in our four outcomes (earnings, employment, public benefit receipt, and education and training). When the quality of a study is high, that means we have strong evidence that the study findings are solely attributable to the program examined. This rating is reserved for study findings from high quality randomized controlled trials with low attrition of sample members. Pathways to Work also reports effects for moderate quality studies, where there is some evidence that the study findings are attributable, at least in part, to the program examined.

There are several reasons that some findings in a study may not be reported:

• Studies may have outcomes that are rated low quality, where there is little evidence that the study findings are attributable to the program examined. Other important factors could have influenced the study findings, and the study did not account for them. Pathways to Work does not report outcomes that are rated low quality.
• If the original study provides findings for multiple measures in a given outcome domain, Pathways to Work prioritizes findings to review and to report following the prioritization process summarized in Exhibit 3.2 of the Protocol for the Pathways to Work Evidence Clearinghouse. For example, if study authors used both surveys and administrative records to assess earnings, Pathways to Work reviewers select two sets of earnings findings for review: one measured using survey data and one measured using administrative data.
• In some cases, Pathways to Work may not have sufficient information to report on the magnitude of study findings. When we can obtain the necessary information from study authors, we calculate an effect size using study-specific data.