Using Big Data For Bigger Impact

The Big Data narrative in the recent years has been predominantly confined to commercial applications. In fact, the who’s who of tech companies are using advanced software tools to understand their users better and thus generate bigger profits.

It is, therefore, a welcome change to note that organisations such as the World Bank are calling upon the experts of the Big Data community and enlisting their help in solving real-world issues related to poverty, development and corruption.

A guiding light
Between March 15 and 17 (this year), around 150 volunteer Big Data experts gathered in Washington DC to discuss in depth a range of social projects requiring their domain knowledge. One of the most innovative projects was the prediction of poverty levels in Bangladesh – not done through questionnaires or accessing bank records, but by tracking the degree of night time illumination in different parts of the country.

By accessing remote sensing satellite data on night time illumination, the team working on the project was able to overlay information gathered separately from census data and figure out if a strong correlation existed between poverty levels and night time illumination.

The results threw up some interesting findings. While the datasets for 2001 indicated a strong correlation, the same trend could not be established for 2005. However, the statistical and geospatial models, built as a result of this pilot effort, could go a long way in becoming a template for studies in other poverty-hit regions across the globe.

Volunteer vision
While organisations like the World Bank and the United Nations Development Programme (UNDP) are backing such data dives, the real impetus for using Big Data for global good is coming from groups like DataKind, which describes itself as a community of pioneering data scientists, visionary change-makers, community builders and social innovators.

When the non-profit organisation Grameen Foundation came up with the innovative idea to equip Ugandan volunteers with mobile phones through which critical information relating to farming could be disseminated to farmers on-field, it didn’t just stop there. The organisation recorded every information search made via mobile phones, the date and the time of the search, as well as the search location.

Through a rigorous analysis of the terabytes of data accumulated, Grameen Foundation found a way to monitor and measure the interactions between the volunteers and the farmers, and determine what was working and what was not.

Under the aegis of DataKind, projects ranging from tracking funding to politicians in the US to collecting accurate pricing data for food items in Africa are being fast-tracked – so that the communities impacted by these data points can improve even further.

Mobilising big resources
This does raise a pertinent question, though. Are these Big Data projects dependent on a big collaborative push from large global entities or can smaller communities across the globe embark on similar initiatives? Seeking help while others are thousands of miles away in Washington DC or Vienna is beyond the resources of most folks working in the social sector.

By definition, most Big Data projects revolve around scale, even when the benefits accrue at the individual level. Take, for instance, the case of Aadhaar – India’s most ambitions unique identification (UID) project. Billed as the world’s largest biometric programme, the project is to provide unique identification numbers or Aadhaar numbers to 1.2 billion people.

The scale gets more impressive when one considers that there are three primary modes of identification built into a single Aadhaar card that is provided to each and every person. These include a photograph, a fingerprint and an iris scan. That means a total of 3.6 billion data objects, if you go by a simple back-of-the-envelope calculation.

It’s no wonder then that the Indian government had to set up a specialised agency, the Unique Identification Authority of India (UIDAI), for this purpose. The government also appointed a former head of a leading Indian IT company to head the project and spent billions of dollars to get things implemented.

The Aadhaar card is expected to be used for every critical function – right from opening bank accounts to receiving government subsidies related to food, fuel and education. Such a Big Bang approach can only be justified on the basis of its multiple uses. However, there are only a handful of organisations in developing countries that can deploy such massive resources. So what happens to the rest?

Big challenge ahead
Ravi Prakash, who wears the twin hats of an engineer at NASA’s Jet Propulsion Laboratory and an IDEO (innovation consultancy firm) fellow, is trying to make a big difference to India’s rural healthcare scenario with the help of Big Data. He is helping the healthcare non-profit World Health Partners’ design systems that can capture each and every health-related interaction in a bid to improve forecasting healthcare issues, enhance healthcare quality and increase operational efficiency. As the IDEO team discovered during its study phase, it requires a whole level of different experts to operate in rural India. And all data point to a difficult but interesting journey ahead.

Image: Thinkstock