I'm a sociologist who studies how police use data. Relying on algorithms can further bias and inequality — but it doesn't have to be that way.

Business Insider

Algorithms are not objective — they're shaped by the social world.TIMOTHY A. CLARY/AFP via Getty Images

  • Sarah Brayne is an assistant professor of sociology at The University of Texas at Austin; prior to joining the faulty at UT-Austin, Brayne was a Postdoctoral Researcher at Microsoft Research.
  • Brayne is also the founder and director of the Texas Prison Education Initiative, a group of faculty and students who volunteer teach college classes in prisons throughout Texas.
  • The following article is based on her forthcoming book, "Predict and Surveil: Data, Discretion and the Future of Policing."
  • In it, she writes that many politicians and reformers have called for more and better data on policing — but it's important to remember that the idea of objective big data isn't true.
  • Algorithms are shaped by the social world — and, dangerously, they can make biases invisible in the form of numbers.
  • But data can help identify non-police functions and maximize public safety, like when a mobile mental health crisis team might be more helpful.

Business leaders and social scientists know: bad data in, bad data out. As the country struggles with demands for systemic change in policing, would-be reformers are suggesting that "data-driven policing" should be part of the solution. It won't solve the problem.

In the weeks since George Floyd was killed by then-Minneapolis police officer Derek Chauvin, protestors, politicians, and pundits have called for an end to policing as we know it. Amid urgent conversations about how to skillfully defund, shrink, or abolish the police, reformers have sought the supposed objectivity of big data. Their logic is to strip discretion from biased front-line officers and replace it with "neutral" "data-driven" policing to solve the all-too-human problems of unjust, violent, and racist policing. Advertisement

Sarah Brayne.Jona Christina Davis

As a sociologist, I have spent nearly a decade studying how police already use big data in American cities. That's hundreds of hours interviewing cops, shadowing crime analysts, and riding in police cruisers to see how officers deploy data in the field.

My on-the-ground approach helped me understand the practice of data-driven policing, including how cops use predictive algorithms and emerging surveillance technologies — many initially designed for military applications — to decide who, when, where, and how to police.

The same politicians and reformers who call for more data, better data, and bigger databases to rein in police brutality hopefully suggest we apply them to any number of other problems in policing: Defunding the police and need to cut costs? Data can help allocate resources most efficiently. Need to reduce racial bias in officer decision-making? Automate it. Want to reduce the categorical suspicion of young Black men and more accurately predict crimes? Try predictive algorithms.

The promise of both efficiency and accountability has already proven nearly irresistible to police departments, yet I have seen firsthand how systems that could help with, say, officer accountability, are not used for that purpose. The bigger problem is that the objectivity of big data is a false premise. Algorithms are shaped by the social world in which they are created and used. That may be great if you're a pharmacy chain trying to predict whether a specific customer might be tempted by a coupon for toothpaste, sent straight to their loyalty card. Advertisement

But algorithms built for social control encode discriminatory laws and past precedents and legacies of segregation and inequality. That's the "bad data." Tell an algorithm that, say, 80% of arrests resulting in charges for cannabis possession come from a certain neighborhood, and the algorithm may tell you to police that neighborhood more closely — not that you have policed it too closely in the past.

In simple terms, predictive algorithms aren't objective. They hold up a mirror to the past to project into the future. If historical biases and police practices inform where and whom the police surveil, they also shape where and from whom cops collect the crime data that is fed into predictive algorithms to generate risk scores. It's a self-fulfilling statistical prophecy.

"Predict and Surveil: Data, Discretion and the Future of Policing."Oxford University Press

Humans decide what data to collect and analyze, about whom, and for what purpose. So, just as individuals carry a range of biases that affect their decisions, an algorithm can become a Trojan Horse: positioned as a gift to society, it actually smuggles in all sorts of biases, assumptions, and drivers of inequality.

What's particularly dangerous is that when the social world is transformed into data points, these biases become invisible. In the words of one police captain, the predictive policing that increasingly directs major American police departments to surveil some areas (and not others) and some people (and not others) are "just math." Who could argue with numbers?

Data will absolutely play an outsized role as this country reckons with its history of racially violent policing. But here's the thing: we can use data to police the police. Because there are already so many data-intensive policing practices at work in American cities, we have a trove of data on police activities. Like the rest of us, police leave digital trails, and those digital trails are susceptible to oversight. Data can show us systematic problems — patterns of officer misconduct and police killings, for instance — that in the past could be dismissed as the actions of a "few bad apples." Advertisement

As we rethink not just the practice of policing, but also the institution of the police (and the outsized role of police unions in protecting and directing this institution), many have rightly noted that law enforcement has, over the last 40 or so years, been charged with all sorts of previously non-police functions. Data truly can help here.

When situations would better be handled by medical or social service professionals, mobile mental health crisis teams, or trained community mediators, algorithms can help us determine how to maximize public safety. These ideas are not new — though there are tons of ways to do that. A word of caution, however: we must be vigilant that "benevolent" interventions do not simply become social control by another name.

Cautiously, we will need to avoid the trap of data worship. Data is not objective. Accountability does not flow automatically from transparency. Police and reformers and abolitionists are all capable of presenting the same data in starkly different ways. But the data web already being used by U.S. police every day can absolutely help us build something better. Rather than the expansion of data-driven policing, we need an enormous, data-informed overhaul of public safety.Advertisement