Data mining for development

Analysis of digital data is a specialised field now, and the multi-billion dollar business of Data Analytics has had significant adoption in the commercial arena lately. It is the development sector that is now waking up to the potential of data mining for analysis and redressals of chronic development woes.
One such initiative, Global Pulse was launched by the United Nations Secretary General in 2009, to use the real time data to better understand the dynamics of development and the crises that reverse it. The need for actionable information was realised after the triple global shocks of food, fuel, and financial crises went largely unnoticed prior to 2007-08, having a hard impact on the global poor and vulnerable.
Traditional statistics and database can track medium-to-long-term trends. But the UN, after decades of institutional experience, has learnt that trend indicators are not much useful when it comes to generating real-time information decision makers need to come up with prompt policy prescriptions for shielding their vulnerable populations from crises.
In a way, the Global Pulse initiative is about turning imperfect, complex, and unstructured data into actionable information, and to leverage that information for a proactive and effective policy response. The project draws on the expertise of other UN agencies, member countries, academic community and private sector. It is important to understand how this all fits into the development equation.
The Pulse teams mandate is not to protect countries against development reversals in these times of increasing global shocks and vulnerabilities. Rather, it is to identify and monitor new sources of real-time information, searching for patterns, validating and interpreting them for policy inputs. This can help policymakers understand the impacts of a crisis on vulnerable populations early on.
A unique innovation to have come out of this project is the Pulse Lab - a sort of an innovation laboratory where analytical and research tools are used for real-time impact monitoring, collaborative analysis and decision-making. The first lab has been established in New York, and the Pulse teams strategy is to create a network of such innovation centres in each UN member country, where analysts and planners from the public, private and development sectors work together.
The Pulse initiative seems high on the UN Secretary Generals agenda, as it is housed in his Executive Office in New York. While more resources are expected to be earmarked for this initiative, there are skeptics who question whether data mining can alter the development situation, especially when strategic decisions will still be taken by the same in-country policymakers. They have a point!
The Pulse team addresses this doubt in its latest report titled Big Data for Development, noting that both the policymakers intent and researchers capacity are crucial to deliver on this front. A lot depends on the institutional and financial support from public sector, the collaboration of private corporations and academia, and the responsible use and sharing of the data for development, it highlights.
It seems that mining the big data, which a whole lot of is available out there, may not be the answer. But it may lead to finding the answers for many of the development questions the policymakers are still grappling with.