Targets, KPIs and why we should be wary

In a 2006 paper, Bevan discusses the problems with targets. In summary these are twofold. One is synedoche - the assumption that the part represents the whole. The other is gaming. Gaming leads to 3 phenomena.

The first is the ratchet effect where things get 'better' year on year, often as a result of setting future targets against benchmarks of previous performance. We see this a lot in healthcare. It is rife in infection control - 95% hand hygiene compliance rates (when the best studies achieve only about 60%) is one example. No hospital wants to look like its hand hygiene is worse than that of the next door hospital.

The second is the threshold effect, where managers alter the system to deliver to the target...and no more. There is no reward for doing better. In the paper, Bevan sites the evidence that ambulance trusts redistributed response centres to urban areas. This had little effect on urban response times, beyond bringing them below an 8 minute threshold, but had a profound effect on rural response times. But as these were relatively small numbers, the effect on the target could be ignored. 

The third type of gaming is manipulation of the output data. There are many ways to do this. Clinical coding is a minefield of variability to be exploited here. And we are all familiar with the stories of A+E trolleys becoming beds. I recently heard about an out of hours GP service that performance manages against time to triage. Very worthy. But clinicians quickly learn they can stop the clock by entering a single full stop into the clinical record, getting the managers off their backs, while they get on with doing the work they were trained to do.

But more worrying is actual distortion of clinical practice. So, for example, surgeons refusing high risk cases. And I know of one laboratory that doesn't load blood cultures if clinical details are vague in order to hit MRSA targets. There is a valid clinical argument here, but it's quite weak, and surely this decision is now open to considerable criticism in the face of the target culture.

Bevan discusses how the target culture in the USSR led initially to large productivity gains over the first couple of decades. But this was followed by stagnation and ultimate failure. Perhaps we see the same in health. Targets initially are well meaning and often focus activity on issues of concern. But quickly, the target becomes the point, and the purpose of the target as an agent of change becomes traduced.

Bevan argues that we can improve things. More random checks perhaps, using more random measures. But ultimately we probably just need more face to face peer led assessment. And for me, this probably leads us away from the comfort of the target, to a more nuanced narrative assessment written in collaboration between assessors and those doing the work.