Performance Measurement? Proceed with Caution

A theme of the “New Management” is to better measure government performance and, in particular, the performance of individual workers. As we computerize more government operations, such performance measurement becomes more feasible and arguably more attractive.

However, an overview essay that I co-authored in a new book on performance measurement and performance management “Improving Public Services: International Experiences in Using Evaluation Tools to Measure Program Performance,” suggests caution in such movement to performance measurement. The argument follows from a cost-benefit analysis.
 

The Cost Side

Even with dropping costs for and wider use of computerization, performance management is far from free. This is true for several reasons.

First, often the measurement itself is expensive. For example, measuring teacher performance requires testing students. Better measurement requires more testing and therefore more time — time that could be used for instruction.

Second, even when operations are computerized, using those systems for performance measurement implies a need for higher quality data. Staff need to enter data with a higher degree of completeness and correctness, all of which takes time. Then, staff entries need to be audited. In the absence of auditing, the temptation to shade the truth — or just lie — is too great. Teachers have inappropriately helped their students with tests used to measure the teachers’ performance. That auditing is expensive.

Third, simple measurement of performance usually is insufficient. Implicitly, performance measurement is usually about comparing performance across workers. If the goal is compare performance across workers, then the measurement system should take steps to assure that differences across workers represent differences in worker productivity — and not something else. In particular, workers assigned “harder” cases will appear worse.

One approach to this challenge is to randomly assign workers to cases. However, doing so gives up the benefits of specialization (e.g., language spoken, type of case) and of worker choice over which types of cases they like to work with. Another approach is statistical modeling (e.g., VAM/Value Added Models in education). However, such statistical modeling often is imperfect and requires detailed information on the cases, such as previous test scores, which often is not available.

Finally, performance management itself can induce undesirable behavior. Workers might try to get “easier” cases, or they might provide less help to their colleagues. When faced with high stakes performance management workers might even act inappropriately or illegally. In a recent banking scandal, to look better on performance measures, workers opened unauthorized bank accounts. Auditing can identify and deter some, but not all, such undesirable or inappropriate behavior. Again, such auditing is expensive.
 

The Benefit Side

Two main arguments exist about the potential benefits of performance measurement. One is that performance measurement allows the agency to understand “what works.” We are dubious about this argument. Understanding what works is the task of evaluation. Doing evaluation properly requires large samples and careful research designs. Performance measurement is a powerful input into such evaluation. However, most informal inferences about what works from ongoing performance measurement data are likely to be wrong!

Second, the private sector uses performance measurement for performance management; i.e., to induce workers to work harder towards organizational goals. Could government do the same? Maybe.
The private sector induces more hours at work, more of those hours on actual work tasks, and more focus of work on company goals with a suite of strong personnel actions. Private sector workers often face a real risk of firing for weak measured performance, large pay increases with promotion where promotion rates differ sharply with measured performance, and large bonuses strongly tied to measured performance.

In contrast, government employment has traditionally been characterized by only weak personnel actions: dismissal is very difficult even when measured performance is weak. Promotion is only loosely related to measurement performance. Pay increments are small. Bonuses are rare, small, and only weakly related to measured performance. None of this is intrinsic to government, but it is the most common case in government.
 

Weighing Costs and Benefits

There is clear evidence that strong performance measurement combined with substantial consequences of measured performance — firing, promotion, bonuses — can improve individual and public agency performance. When an agency has the ability and the will to manage based on measured performance, measuring performance often will be worth doing.

However, measuring performance well enough to use it in personnel management is not inexpensive. Doing it well requires substantial resources and often reorganizing the way work is done so that measured performance is strongly related to true individual performance. Unless the agency has the authority and the will to fire workers, to vary promotion with performance, and to vary the size of bonuses, the costs of performance measurement may not be worth bearing. Conversely, the lower the cost of measuring performance, the more attractive it will be to do so and then to use that measurement in personnel management.

So consider performance measurement, but carefully!

References
  • Baehler, Karen and Jacob Klerman. 2017. “Measuring and Managing Farther Down the Logic Model”, in Modern Performance Measurement around the World: Unifying Theories and Cross-National Applications? Oxford University Press.
  • Besharov, Douglas, Karen Baehler, and Jacob Klerman, eds. 2017. Modern Performance Measurement around the World: Unifying Theories and Cross-National Applications? Oxford University Press.
  • Klerman, J.A. 2005. “Performance Measurement in Government” in High-Performance Government: Structure, Leadership, and Incentives, ed. Robert Klitgaard and Paul Light. Santa Monica, CA: The RAND Corporation.
Comments
Blog post currently doesn't have any comments.
Leave comment



 Security code