The ESER brief focuses on a quantitative synthesis—known as a meta-regression—of the research evidence across all the studies included in ESER to examine which employment services are most effective. A meta-regression analyzes outcomes from different studies rather than outcomes from different people. The ESER team searched the literature for relevant research published from 1990 to mid-2014 and then screened for eligible studies to review—those that used randomized controlled trials or comparison group designs. The reports were then rated by the research team based on their rigor - random controlled studies, comparison group designs and literature reviews - to determine which studies and associated data to include in the ESER review.
The authors describe their approach as follows, "We estimated the relationships between the outcomes from the ESER-reviewed studies and the services offered and populations targeted by each intervention to explore what combination of strategies works for employment-related outcomes, which individual strategies are associated with the greatest improvement in impacts, and which strategies work best for specific populations."
The ESER review sought to answer the following three questions using the meta-regression approach:
1. What works in general? Which interventions and employment or training strategies improve outcomes for low-income workers overall?
2. What works for particular outcomes? Do some interventions and strategies have different effects for different outcomes? For example, job search assistance may help people find jobs but not ensure their independence from public assistance.
3. What works for particular workers? Which strategies are most effective for particular types of low-income workers? For instance, basic skills training may be more effective for those lacking a high school diploma than it is for those who have a diploma or GED.
Major Findings & Recommendations
This brief suggests that combining different employment and training strategies, as most interventions do, can have positive and significant impacts on outcomes for low-income individuals. However, these combinations do not fully explain the effectiveness of an intervention: the effect of an intervention is more than the sum of the effects of that intervention’s strategies. In this context, implementation and other idiosyncratic factors become all the more crucial to our understanding of effectiveness.
Key findings suggest that employment and training strategies and interventions demonstrate effectiveness both overall and for specific outcome types. The following highlights are noted in the brief:
- Nineteen of the 93 unique interventions in the ESER database caused significantly favorable impacts, while one caused significantly unfavorable impacts.
- Most strategies are associated with modest positive effects.
- No single strategy on its own is associated with substantial gains.
- Although no strategy is statistically significantly more effective than any other, whether in general or for specific populations or outcome types, all strategies are generally effective—that is, they are associated with increased impacts.
- Four strategies, financial incentives and sanctions, education, work experience, and training, have over a 90 percent chance of improving outcomes across population and outcome types. These four employment strategies are associated with marginally better—between 2 and 3 percent increases— outcomes for some outcome types. Work experience is associated with improved impacts on short-term independence from public assistance; education is associated with improved impacts on both education and training outcomes and long-term independence from public assistance; and, financial incentives and sanctions are associated with improved impacts on short-term employment rates.