Rhode Island sought to accomplish the first goal by mapping career pathway frameworks, aligning funding and policies, creating tools to support jobs/pathways, integrating supportive services for job seekers and workers, and creating more integration between public and private workforce systems. Objectives within the second goal were the piloting of a new business model at two One-Stop centers and various locational sites and implementing that model, which included work readiness training, career coaching, and work experience.
The On-Ramps to Career Pathways (ORCP) project was an initiative that included both a systems change component and a participant-level pilot (the On-Ramps Pilot). For the systems change component, a goal was to reallocate resources and reconfigure policies to support the implementation of a new set of services to better meet the needs of workforce participants and employers. This component involved working in partnership with a number of state agencies and other partners to align state funding, policies, and support services to strengthen the state workforce system in the six areas: career pathways, aligned funding, support services, work readiness, work experience, and common performance measures. For the On-Ramps Pilot, ORCP aimed to develop and implement a new set of services for workforce participants to be delivered through American Job Centers (AJC) and other local partners, to improve low income and low-skilled participants’ career focus and job readiness, reflected in improved employment, wages, and job retention. The Pilot, as designed, included three components: work readiness training, work experience, and career coaching. In the work readiness component, participants attended a two-week, 30-hour course using curricula procured from national vendors, which included skill assessment, career exploration activities, career planning, and resume development. At the end of the training, participants who passed the course were granted the National Career Readiness Certificate (NCRC). Although the Pilot was to include the work experience and career coaching components, these services were not fully implemented as planned.
The evaluation of the ORCP began with the start of the project and followed the project and its results. It involved three components: implementation, outcomes, and cost studies.
(This report is 105 pages long including appendices.)
Major Findings & Recommendations
The evaluator concluded that the ORCP project faced numerous implementation challenges, which prevented it from accomplishing its goals. While a competent and dedicated team was assembled to implement ORCP, the staffing capacity, management tools, and overall support for the ORCP initiative were insufficient to meet the ambitions of the project. The evaluation yielded these key findings:
- The ORCP achieved important milestones in its systems change goal, particularly pertaining to career pathways, which has continued to be a focal point for Rhode Island.
- The pilot was not implemented as planned and did not reach its targets for participation. The pilot faced delays and challenges, resulting in a total of 599 served rather than 1,000.
- The work readiness feature of the pilot dominated in the implementation and was well received by participants. A high percentage of participants received National Career Readiness Certificate (NCRC) recognition at the bronze, silver, or gold levels.
- The On-Ramps pilot, overall, did not result in improved outcomes for participants over the comparison group of One-Stop customers. On-Ramps participants had better employment retention two quarters after program exit than the comparison group, but they did not achieve better outcomes in employment or wage gain.
- The pilot, given the outcomes and its higher price tag, was not cost-effective. The cost of the On-Ramps pilot was approximately twice that of the standard One-Stop services per participant.
- The staffing capacity, management tools, and community of support for the ORCP initiative were insufficient to the ambitions of the project.
Based on the implementation study findings and the related cost and outcomes findings, the team provides the following recommendations:
- Require evidence in proposals that performance targets are reasonable and attainable.
- Provide a planning period to solidify commitments and resources and to build the organizational capacity to conduct the work.
- Require that proposed projects develop an implementation plan and subject the plan to analysis before committing funds.
- Ensure the project leadership fully understands the conditions for a rigorous outcomes analysis.
- Monitor ongoing project performance from a substantive perspective and do so in consultation with the third-party evaluators.