CareerSource Florida, the state of Florida's workforce system's policy making board, designed the PFM program as a strategy to reward the performance of the state’s 24 local workforce development boards (LWDBs) across seven performance metrics. The aim was to motivate LWDBs to increase efficiency and effectiveness, ultimately resulting in better outcomes for clients (such as increased employment and wages). The seven performance metrics fell into three categories of “placement” and “exit” metrics, which focus on job seekers, and “business” metrics which focused on businesses served by the CareerSource Florida network. The study of the effectiveness of the program therefore aimed to assess the difference in employment and wage outcomes exhibited by clients of the state workforce system before and after the statewide innovation. Evaluation of the PFM program included an implementation, an outcomes, and a cost study.
The report is 134 pages long including attachments.
Major Findings & Recommendations
The report authors identified the following lessons learned:
Key findings of the implementation study found that the rollout of the project deviated from the original plan due to several external factors. Turnover of key staff, initial internal delays, and the effects of two natural disasters all contributed to the implementation challenges of the project. A challenge identified by the study indicated that the lag in accessing certified wage data adversely affected the PFM model. The PFM theory of change dictates that boards change their behavior based on performance feedback. In the case of wage data, feedback arrives too late to impact operational decision making.
The outcomes study produced mixed results, finding marginal positive effects on employment outcomes, marginal negative effect on wage outcomes and small positive effects on employment for those workforce program participants who were unemployed at enrollment. The evaluation suggests that the period for the evaluation may have been too short to fully assess results, especially those on labor market outcomes. Despite the implementation challenges, the evaluator found that there is evidence of promise for the model. Stakeholders reported that having learned lessons around the value of planning, partnerships and continuous improvement, there was potential to engage the PFM program or a revised version going forward.
The cost study calculated the costs for three distinct phases that occurred during implementation of the PFM program: development, startup implementation, and ongoing implementation.
The report authors provided several recommendations including the following highlights:
- Given the complexity and scope of the project, having necessary staff (both technical and non-technical) aligned to the key functional requirements of the model was important.
- Providing boards with access to up-to-date data and the ability to drill down in those data was key for both board understanding of the PFM and the metrics, as well as for the motivation to respond to data and enact change.
- The funding provided must be substantial enough to motivate participation for boards of all sizes to participate in and engage with the model.
- Obtaining and carefully integrating stakeholder feedback from the beginning of the project facilitates the creation of a model that is relevant, affects decision making as intended, and is sustainable in design.