Study Details | Pathways to Work

24966-Study of STEP Forwar

Study quality: High

High

Reason for the rating: This study received a high study quality rating because it is a randomized controlled trial with low attrition, suggesting the findings can be attributed to the program.

Program: STEP Forward

Characteristics of study participants:

Adults (age 25+), Unemployed

Services provided:

Case management, Childcare and/or early education, Employment coaching, Job development, Job placement, Job readiness, Job search assistance, Mental health services, Soft skills training, Subsidized employment/transitional jobs, Transportation assistance

Setting: Urban only

Methods and Standards Protocol version used for review: Version 1.0

Learn about our protocol

Study design

Design:

Randomized controlled trial

Study group formation:

The Human Services Agency (HSA) of San Francisco, which operated the program, identified eligible participants who had been unemployed for the previous three months and had no more than six months of earned income in the previous two years. The program initially focused on California Work Opportunity and Responsibility to Kids clients (who had timed out or were exempt from work requirements) and unemployment insurance exhaustees (who had already received unemployment benefits for the maximum number of weeks and had income below 200 percent of the federal poverty level). HSA also recruited from additional populations to create a larger sample, including people enrolled in the CalFresh (California's Supplemental Nutrition Assistance Program) Employment and Training program, County General Assistance recipients who did not participate in Personal Assisted Employment Services, and former JOBsNOW! participants who were state Temporary Assistance for Needy Families (TANF) or General Assistance clients and had received two or more subsidized employment job placements through JOBsNOW! but no unsubsidized employment. Interested people attended prescreening sessions with HSA staff to confirm they were eligible before being randomly assigned. A total of 837 people were randomly assigned to the program group (421) or the comparison group (416) from November 2012 to March 2015.

Time period of study:

Evaluators randomly assigned people into groups for the study from November 2012 to March 2015. The study reports impacts up to four years after random assignment.

Primary outcome domains examined:

Increase short-term earnings, Increase long-term earnings, Increase short-term employment, Increase long-term employment, Increase education and training

Other outcome domains examined:

Economic well-being and personal well-being

Study funded by:

ACF, U.S. Department of Health and Human Services

Results

View table definitions

Scroll to the right to view the rest of the table columns

Outcome domain	Measure	Timing	Study quality by finding	Comparison group mean	Program group mean	Impact	Units	Sample size
Increase short-term earnings	Annual earnings	Year 1	High	6,912.00	8,520.00	1,608.00	2015 dollars	811
Increase long-term earnings	Annual earnings	Year 4	High	16,580.00	19,521.00	2,941.00	2018 dollars	811
Increase short-term employment	Currently employed	12 months	High	43.30	50.00	6.70	Percentage points	556
Increase short-term employment	Employed for four consecutive quarters	Year 1	High	26.50	25.30	-1.20	Percentage points	811
Increase short-term employment	Ever employed, quarterly	Quarter 5	High	55.40	62.10	6.70	Percentage points	811
Increase short-term employment	Number of quarter employed	Year 1	High	2.00	2.10	0.10	Quarters	811
Increase long-term employment	Ever employed, annual	Year 4	High	72.00	75.00	3.00	Percentage points	811
Increase education and training	Earned a professional license or certification	12 months	High	11.00	11.40	0.40	Percentage points	556
Increase education and training	Received high school diploma or GED	12 months	High	0.00	0.40	0.40	Percentage points	556

Short-term outcomes are those measured 18 months or fewer after participants are first offered services. Long-term outcomes are those measured between 18 months and 5 years after participants are first offered services. Very long-term outcomes are those measured 5 years or more after participants are first offered services.

Means are not displayed when not reported or not aligned with the impact estimates. For example, if the impact estimate is regression-adjusted but only unadjusted means are reported in the study, the unadjusted means are not displayed as they are not aligned with the adjusted impact estimate.

Impact estimates are not shown when the estimate units cannot be converted to natural units.

Findings quality key:

High
Moderate

Study quality terms?

The findings quality describe our confidence that a given study’s finding is because of the program. We do not display findings that rate low.

Findings key:

A moderate-to-large favorable finding that is unlikely to be due to chance
A moderate-to-large favorable finding that might to be due to chance
A small favorable finding that is unlikely to be due to chance
A small favorable finding that might be due to chance
A favorable finding that is unlikely to be due to chance, but we cannot determine the standardized effect size
A favorable finding that might be due to chance, but we cannot determine the standardized effect size
A moderate-to-large unfavorable finding that is unlikely to be due to chance
A moderate-to-large unfavorable finding that might to be due to chance
A small unfavorable finding that is unlikely to be due to chance
A small unfavorable finding that might be due to chance
An unfavorable finding that is unlikely to be due to chance, but we cannot determine the standardized effect size
An unfavorable finding that might be due to chance, but we cannot determine the standardized effect size
A finding that is unlikely to be due to chance, but we cannot determine the standardized effect size or direction
A finding of no effect that might be due to chance

Favorable findings are impacts in a direction that is socially desirable. Statistical significance indicates the likelihood that a study finding is due to the program, rather than due to chance. Findings are described as moderate-to-large if the program produced a sufficiently large change in the outcome. An impact receives this designation if the standardized effect size is equal to or greater than 0.25 or equal to or less than -0.25.

Sample characteristics

The study sample included CalFresh recipients (37 percent), recipients of TANF who had exhausted their benefits (27 percent), recipients of unemployment insurance who had exhausted their benefits (20 percent), and other people deemed eligible for services. The majority (72 percent) were female, 42 percent were Black, 23 percent were Asian, 19 percent were Hispanic, and 9 percent were White. The average age was 41. At the time the study began, 60 percent were never married, and 68 percent had children who were minors. Nearly 33 percent had worked fewer than six months in the last three years, and 19 percent did not have a high school diploma.

Age

Mean age	41 years
Young adults (age 16-24)	5%
Adults (age 25+)	95%

Sex

Female	72%
Male	28%

Race and ethnicity

Black or African American	42%
White, not Hispanic	9%
Hispanic or Latino of any race	19%
Asian	23%
More than one race	4%
Unknown, not reported, or other	3%

The race and ethnicity categories may sum to more than 100 percent if the authors reported race and ethnicity separately; in these cases, we report the category White, rather than White, not Hispanic.

Family status

Married	21%
Parents	68%

Employment and public benefit status

Were unemployed	100%
Public benefits recipients	64%
Had low incomes	100%
Cash assistance recipients	27%

Education

Had a high school diploma or GED	72%
Did not have a high school diploma or GED	28%
Postsecondary credential	22%
English learners	6%

Specific employment barriers

Had a disability	10%
Had a mental illness	20%

Program implementation

Implementing organization:

HSA of San Francisco

Program history:

The JOBsNOW! program, which was only available to California TANF recipients with dependent children and incomes below 200 percent of the federal poverty level, launched in 2009 and was funded by the American Recovery and Reinvestment Act. STEP Forward was an extension of JOBsNOW! and expanded eligibility for wage subsidies.

Program services:

The JOBsNOW! STEP Forward program served people with low incomes who were experiencing unemployment. Participants had access to interviews for subsidized employment opportunities, individualized career counseling, job interview preparation, job development services, and case management. Participating employers received a subsidy of up to $5,000 over five months for each program participant that employers hired at the prevailing market wage for at least 25 hours per week.

Mandatory services:

None

Comparison services:

The comparison group could receive services as usual in the community, such as assistance with housing, transportation, and job search, but did not have access to subsidized employment opportunities or other services provided by STEP Forward.

Service receipt duration:

Wage subsidies could last up to five months.

Program funding:

ACF, U.S. Department of Health and Human Services

Cost information:

The average cost of providing the intervention was $5,521 per participant. The average cost of providing comparison group services was $2,392 per person. The net intervention cost, or difference between the two, was $3,129 per person. This represents how much more it cost to provide intervention services compared to comparison services.

These figures are based on cost information reported by study authors. The Pathways to Work converted that information to a single amount expressed in 2024 dollars; for details, see the FAQ. This information is not an official price tag or guarantee.

Study publications

Walter, Johanna, David Navarro, Chloe Anderson, and Ada Tso (2017). Testing rapid connections to subsidized private sector jobs for low-income individuals in San Francisco: Implementation and early impacts of the STEP Forward program, OPRE Report #2017-103, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Available at https://www.mdrc.org/sites/default/files/2017_sted_sf_FR.pdf.

Webster, Riley (2019). Cost analysis of the STEP Forward program, OPRE Report #2019-109, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Available at https://www.mdrc.org/sites/default/files/step_forward_cost_analysis_nov_2019.pdf.

View the glossary for more information about these and other terms used on this page.

Pathways to Work refers to programs by the names used in study reports or manuscripts. Some program names may use language that is not consistent with our style guide, preferences, or the terminology we use to describe populations.

Study of STEP Forward

24966-Study of STEP Forwar

Study design

Design:

Study group formation:

Time period of study:

Primary outcome domains examined:

Other outcome domains examined:

Study funded by:

Results

Sample characteristics

Age

Sex

Race and ethnicity

Family status

Employment and public benefit status

Education

Specific employment barriers

Program implementation

Implementing organization:

Program history:

Program services:

Mandatory services:

Comparison services:

Service receipt duration:

Program funding:

Cost information:

Study publications