by Richard Martin Oxman and Crown Publishing colleagues
We live in the age of algorithm. Increasingly, the decisions that affect our lives — where we go to school, whether we get a car loan, how much we pay for health insurance — are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness. Everyone is judged according to the same rules, and bias is eliminated.
But, as Cathy O’Neil reveals in (what is described as) an “urgent and necessary” new book according to quite a few academics I know, the opposite is true. The models being used today are opaque, unregulated, and not contestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code, or any other of a number of variables), he’s then cut off from the kind of education that could pull him out of poverty, and vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” O’Neil describes that dark side of Big Data in convincing detail, and it begs to be addressed.
She exposes the black box models that shape our future, both as individuals and as a society. These “weapons of math destruction” score teachers and students, sort resumes, grant (or deny) loans of all kinds, evaluate workers, target voters, set parole and monitor our health. Big Data is a Big Deal by any standards.
O’Neil, however, calls on modelers to take more responsibility for their algorithms, and she asks policy makers to regulate their use. And suggests that in the end it’s up to us to become more savvy about the models that govern our lives. All of which makes sense on one level, but a very superficial level. What she’s advocating is common sense advice, of course, and it’s certainly politically correct. But it won’t cut the muster in making a difference. Not alone.
It’s very common to hear activists chant the mantra, “Speak truth to power.” But Power already knows the truth, has known it for quite some time. And Power is quite in control of what models will be employed in gathering data, anyway. Obviously, we can count on the powers that be to embrace self-serving paradigms. Ones that do not stand a chance at bringing about a more just or peaceful world. Continue to increase inequality and threaten democracy by design.
We should, yes, become more saavy, encourage modelers to self-educate and make use of better tools, as we pressure policy makers to do the right thing.
But we must secure significant reins of decision-making capacity ourselves. And the foundation for that must be the kind of association with trustworthiness that’s spotlighted in a recent article by Rachel Olivia O’Connor. To deal effectively with Big Data-induced inequality, and to terminate the threats to democracy which are proliferating exponentially, we must do more than become adept, do more than have others adopt different means of measurement, and — above all — must step back from making our success contingent upon the cooperation of career politicians; some are better than others, but none are what we need at this juncture.
The issues cited in this article are, unquestionably, of dire importance. But they are not the most critical issues of our time. That said, ALL demand that a new breed be at the helm.
That kind of leadership would, among other things, help us to question the educational momentum in this country (the value of school, as things stand), consider alternatives to an embrace of car culture, and have us address the competency of medical professionals along with what they charge for their services.
Such leadership would give us a collective chance to create the watershed in history which is now necessary.
