Metrics

Perhaps an eccentricity associated with my profession, minus an analytical review, and that to a detailed statistical one, it is very unlikely that I might agree with any argument; after all numbers don't lie. And with the advent of technology, this tendency to analyse till death has become even more pronounced; if something can be quantified one way, it can definitely be quantified from up down left and right. And all this was in spite of my personal view about detailed disclosures in financial statements of corporations under the International Financial Reporting Standards; no one reads them, let alone understand those disclosures, and hence all this analysis and information is useless.
But as they say you live and learn. Not everything that can be counted counts and not everything that counts can be counted; a dictum I was reminded of by Jerry Muller in his book "The Tyranny of Metrics"; definitely a game changer for me, in the context of public services, which was the primary focus of the book.
I suppose vis-à-vis the private sector, numbers, and that too specifically the bottom line, will continue to remain pertinent in the foreseeable future; nothing else matters to the "Seth". But how do you measure public services; are public schools imparting quality education necessary for equipping labour with skills necessary for future economic growth; are public health facilities utilizing scarce public finds in the most efficient manner; is the police really doing its job. Admittedly, everyone is expected to answer in the negative for at least the last question, but there is likelihood, even if remote, that such concerns are the cause of sleepless nights for someone somewhere in the State hierarchy.
And what else but business metrics, a quantifiable measure that is used to track and assess the status of a specific activity or process, could provide the solution; except as Muller correctly points out, a public school, a public hospital and the police service are not businesses. And that is followed by his most critical insight in the book; that which can be measured gets gamed. Curiously, while I might have subscribed to the view that economic indicators can be gamed and are gamed, it never occurred to me that public sector measurement indicators could easily be gamed too!
A smart phone App might confirm teacher's attendance via GPS location, however it cannot provide evidence that the teacher is actually "teaching". A testing system to collect data about teaching quality and student learning is indeed useful, but the risk is that the multiple aims of education will be neglected for teaching to the test. Further, the schools are likely to restrict enrolment to the brightest; those more in indeed of education will be ignored completely. And apparently, recent research abroad, rather amusingly, has also failed to prove the nexus between teacher incentives and learning.
If doctors will be assessed on the basis of their success rates, they might chose to ignore any cases which might impact their standings negatively; terminal cases will not get treatment at all. Excessive measurement in the case of hospitals has the dual effect of wasting more resources on preparing data for such measurement and ignoring or misreporting cases simply to meet the standards. Again a smart phone App can confirm attendance, but cannot provide any feedback about the quality of services. Perhaps an automated feedback system might be a solution; however, the likelihood of an illiterate population risking honest feedback about public services is rather remote in a developing country.
In the case of police, how data is gamed does not require rocket science; firstly just don't register the FIR, or if it does get registered, massage the statistics. Perhaps data about police efficiency is not even relied upon by the State itself! "A degree of legitimate concealment is necessary to maintain the state and its democratic institutions", Moshe Halbertal. Unfortunately, perhaps, developing countries have given a new definition to the word "legitimate", for this purpose!
While I am grateful to Muller for all of the insight above, we are left with the predicament; how do you measure and monitor public services? And I also agree with the dictum that if you focus on pay for performance you will only get mercenaries. From my experience, the commitment level and dedication required for public services, and charitable services, are a class apart from the skill set appreciated in the private sector. Perhaps the solution is to focus on selecting and hiring the right people; except that in itself will require long term dedication and commitment absent from Governments which come in for a five year period.
The suggestion is not to desist from using the metric system developed to assess the adequacy and efficacy of public services, except that over reliance thereto is perhaps fraught with danger of misinterpreting the results or gaming; which those at the helm of affairs should remain weary off. Simultaneously, a focus on getting the right people for the job, and not overly nudging them in the wrong direction should be the long-term strategy.
(The writer is a chartered accountant based in Islamabad. Email: [email protected])