Study Details | Pathways to Work

26582-Study of Breaking Ba

Study quality: High

High

Reason for the rating: This study received a high study quality rating because it is a randomized controlled trial with low attrition, suggesting that findings can be attributed to the program.

Program: Breaking Barriers

Characteristics of study participants:

Adults (age 25+), At least a high school diploma or equivalent, Disability(ies)

Services provided:

Case management, Employer-focused retention strategies, Employment coaching, Job development, Job placement, Job readiness, Job search assistance, Post-employment participant follow-up

Setting: Urban only

Methods and Standards Protocol version used for impact review: Version 2.0

Learn about our protocol

Cost: This study includes cost information reviewed by Pathways to Work (under the cost standards in Methods and Standards Protocol Version 2.0).

Study design

Design:

Randomized controlled trial

Study group formation:

Individuals were recruited for the study if they (1) were at least 18 years old; (2) were a resident of San Diego County in California; (3) had a self-identified mental or physical disability; (4) had low income; (5) were not working or were underemployed; and (6) were a participant of California's TANF program, California's vocational rehabilitation service agency, or a San Diego County behavioral health service program. Eligible individuals were randomly assigned to the Breaking Barriers program group or to a comparison group. A total of 1,061 individuals were randomly assigned, with 528 individuals assigned to the program group and 533 individuals randomly assigned to the comparison group. Study analyses excluded veterans who received Breaking Barriers services from the analysis to comply with federal funding requirements and individuals who reported limited English-speaking abilities at the time of enrollment.

Time period of study:

The study ran from January 2017 to December 2019. Study enrollment ran from January 2017 to November 2017. Data collection occurred from January 2017 to December 2019, with outcomes measured using a follow-up survey administered an average of 15 months after random assignment and administrative data over a two-year follow-up period.

Primary outcome domains examined:

Increase short-term earnings, Increase long-term earnings, Increase short-term employment, Increase long-term employment

Other outcome domains examined:

Physical health status; mental health status

Study funded by:

This study was funded by a grant from the U.S. Department of Labor, Employment and Training Administration and the U.S. Department of Health and Human Services, Administration for Children and Families (Contract Number HHS-P233201500059I).

Results

View table definitions

Scroll to the right to view the rest of the table columns

Outcome domain	Measure	Timing	Study quality by finding	Comparison group mean	Program group mean	Impact	Units	Sample size
Increase short-term earnings	Annual earnings, NDNH data	Year 1	High	6,427.00	8,807.00	2,380.00	2018 dollars	1,061
Increase short-term earnings	Total earnings over follow-up period, survey data	Month 15	High	10,973.00	11,335.80	362.87	2018 dollars	648
Increase long-term earnings	Annual earnings, NDNH data	Year 2	High	8,692.00	10,276.00	1,584.00	2019 dollars	1,061
Increase short-term employment	Currently employed, survey data	Month 15	High	57.00	60.90	3.90	percentage points	648
Increase short-term employment	Ever employed, annual, NDNH data	Year 1	High	61.00	68.00	7.00	percentage points	1,061
Increase short-term employment	Number of months employed, follow-up period, survey data	Month 15	High	6.20	6.70	0.50	months	648
Increase long-term employment	Ever employed, annual, NDNH data	Year 2	High	60.00	61.00	1.00	percentage points	1,061
Increase long-term employment	Number of quarters employed, follow-up period, NDNH data	Quarter 8	High	3.70	3.90	0.20	quarters	1,061

Short-term outcomes are those measured 18 months or fewer after participants are first offered services. Long-term outcomes are those measured between 18 months and 5 years after participants are first offered services. Very long-term outcomes are those measured 5 years or more after participants are first offered services.

Means are not displayed when not reported or not aligned with the impact estimates. For example, if the impact estimate is regression-adjusted but only unadjusted means are reported in the study, the unadjusted means are not displayed as they are not aligned with the adjusted impact estimate.

Impact estimates are not shown when the estimate units cannot be converted to natural units.

Findings quality key:

High
Moderate

Study quality terms?

The findings quality describe our confidence that a given study’s finding is because of the program. We do not display findings that rate low.

Findings key:

A moderate-to-large favorable finding that is unlikely to be due to chance
A moderate-to-large favorable finding that might to be due to chance
A small favorable finding that is unlikely to be due to chance
A small favorable finding that might be due to chance
A favorable finding that is unlikely to be due to chance, but we cannot determine the standardized effect size
A favorable finding that might be due to chance, but we cannot determine the standardized effect size
A moderate-to-large unfavorable finding that is unlikely to be due to chance
A moderate-to-large unfavorable finding that might to be due to chance
A small unfavorable finding that is unlikely to be due to chance
A small unfavorable finding that might be due to chance
An unfavorable finding that is unlikely to be due to chance, but we cannot determine the standardized effect size
An unfavorable finding that might be due to chance, but we cannot determine the standardized effect size
A finding that is unlikely to be due to chance, but we cannot determine the standardized effect size or direction
A finding of no effect that might be due to chance

Favorable findings are impacts in a direction that is socially desirable. Statistical significance indicates the likelihood that a study finding is due to the program, rather than due to chance. Findings are described as moderate-to-large if the program produced a sufficiently large change in the outcome. An impact receives this designation if the standardized effect size is equal to or greater than 0.25 or equal to or less than -0.25.

Sample characteristics

Study participants were individuals with disabilities and low income who were unemployed or underemployed. Most participants had a mental illness (86 percent) and had at least high school diploma or GED (81 percent). A majority of participants were female (54 percent). About one-third of participants (34 percent) had a substance use disorder.

Characteristics may sum to greater than 100 percent due to rounding.

Age

Mean age	40 years
Young adults (age 16-24)	12%
Adults (age 25+)	88%

Sex

Female	54%
Male	46%

Race and ethnicity

Black or African American	15%
White, not Hispanic	40%
Hispanic or Latino of any race	35%
Unknown, not reported, or other	10%

The race and ethnicity categories may sum to more than 100 percent if the authors reported race and ethnicity separately; in these cases, we report the category White, rather than White, not Hispanic.

Family status

Married

18%

Employment and public benefit status

Were employed	42%
Were unemployed	58%
Cash assistance recipients	28%

Education

Had some postsecondary education	21%
Had a high school diploma or GED	81%
Did not have a high school diploma or GED	19%
English learners	17%

Specific employment barriers

Had a disability	100%
Had a mental illness	63%
Had a substance abuse disorder	34%
Were refugees	8%

Program implementation

Implementing organization:

Breaking Barriers was designed and overseen by the San Diego Workforce Partnership (the county workforce investment board), which contracted with American Job Centers to provide services. Three job centers were operated by nationwide private providers and one was operated by a local school district.

Program history:

Breaking Barriers started a year prior to the study, in January 2016, and operated through June 2018.

Program services:

Breaking Barriers provided Individual Placement and Support (IPS) services within American Job Centers to eligible individuals with low income who had a self-identified disability. IPS services included career counseling, job search assistance, job development, personalized benefits counseling, referrals to supportive services, and follow-along support after individuals found a job. Employment specialists who provided IPS services maintained small caseloads in order to provide participants with individualized support.

Mandatory services:

None

Comparison services:

Individuals assigned to the comparison condition received a list of other publicly available resources in the community.

Service receipt duration:

Participants received services for an average of 10 months.

Program funding:

This program was funded by a grant from the U.S. Department of Labor’s Workforce Innovation Fund.

Study publications

Freedman, Lily, and Megan Millenky (2022). Two-year findings from the evaluation of Breaking Barriers: An individual placement and support (IPS) program. Technical Appendix. OPRE Report 2022-35. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Available at: https://acf.gov/sites/default/files/documents/opre/evaluation-breaking-barriers-technical-appendix-mar-2022.pdf

Freedman, Lily, and Megan Millenky (2022). Two-year findings from the evaluation of Breaking Barriers: An individual placement and support (IPS) program. OPRE Report 2022-35. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Available at: https://acf.gov/sites/default/files/documents/opre/evaluation-breaking-barriers-mar-2022.pdf

Freedman, Lily, Sam Elkin, and Megan Millenky (2019). Breaking Barriers: Implementing individual placement and support in a workforce setting. New York, NY: MDRC. Available at: https://www.mdrc.org/publication/breaking-barriers

View the glossary for more information about these and other terms used on this page.

Pathways to Work refers to programs by the names used in study reports or manuscripts. Some program names may use language that is not consistent with our style guide, preferences, or the terminology we use to describe populations.

Study of Breaking Barriers