Skip to Content

Book ,

A Microsoft researcher confronts how companies shape what we see and say online

Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media

Tarleton Gillespie
Yale University Press
2018
296 pp.
Purchase this item now

On 1 March 2018, Twitter announced a new initiative aimed at measuring and evaluating the platform’s “conversational health,” that is, how well it promotes lively conversation and critical thinking while also minimizing the social impact of abuse, spam, and manipulation. The initiative is one piece of a larger effort by social media companies both to respond to the proliferation of disinformation and propaganda exposed in the aftermath of the 2016 U.S. presidential election and to attend to ongoing issues of sexist, racist, and homophobic abuse and harassment.

In framing the initiative as a matter of “health,” the company put a therapeutic spin on much longer–standing practices for evaluating, filtering, and shaping what we can see and say online. As Tarleton Gillespie captures it in his new book Custodians of the Internet, “[t]he concerns around political discourse and manipulation on social media platforms feel like the crest of a larger wave that has been breaking for a few years now, a broader reconsideration, on so many fronts, of social media platforms and their power in society.”

In the book, Gillespie, a principal researcher at Microsoft Research New England and an adjunct associate professor at Cornell University, goes beyond a mere account of the tools and practices employed by social media companies to address problems of harassment, obscenity, and hate speech on their platforms. He also aims to capture just what is at stake in debates over online expression—from the consequences of being “censored” online to the fate of social and democratic norms in the face of easily manipulable news feeds.

Although these questions have taken on a new urgency, they are not wholly new. As the opening chapters of Custodians make clear, content moderation is endemic to exchanging information over the internet. There has never been a time when policy, social context, or technical realities have not informed the possibilities and limits of online communication.

Despite this long history, the work of moderation has become more pointed, and more challenging, in the 21st century. The explosion of platforms for expression—from message boards and chat rooms to large-scale social networking sites—and the increasing pervasiveness of those platforms in social, political, and economic life demand that we “reconsider what platforms are, and ask new questions about their power in society.” To this end, more than half the book is dedicated to demystifying platforms and their politics, making visible the human labor, internal policies, and automated tools that make content moderation at the scale of millions (or, in some cases, billions) of users possible.

In our current geopolitical moment, it would be easy to criticize parts of Gillespie’s account as frustratingly apolitical. There are times when the reader might reasonably expect him to be more cynical about the relationship between content moderation, the collection of massive amounts of user data, and platforms’ business models. As he himself notes, the mechanisms of moderation “cannot be disentangled from the economic imperatives that drive moderation too.”

But what Gillespie sacrifices in critical detours early on, he gains in expository efficiency. He moves quickly between topics that merit entire volumes themselves, from cataloging the social and technical means by which platforms moderate to capturing the lived experience of users under sometimes hostile systems. (The struggle of mothers attempting to share breastfeeding photos on Facebook, only to be thwarted by overly prudish or actively hostile editorial guidelines, is one example of the latter and features prominently in the middle of the book.)

CJG - TECHNOLOGY/ALAMY STOCK PHOTO

Internet platforms work hard to maintain the illusion of being neutral facilitators of content.

These criticisms, however, are matters of degree, not category. As with the filtering mechanisms described in the book, readers from different backgrounds might calibrate the relative importance of various factors differently. Some might prefer to foreground the economics of content moderation while downplaying concerns over censorship and freedom of expression. Others might disfavor talk of algorithms and automation and instead emphasize the efforts of human laborers—the severely underpaid and overtaxed “clickworkers” who daily must wade through the internet’s most toxic content. That the book is able to fairly account for these myriad and far-ranging dimensions of moderation is a credit to Gillespie’s detailed work.

For the reader looking for a more trenchant analysis, Gillespie’s final two chapters move in this direction. As Gillespie makes clear, online platforms work hard to keep their moderation practices invisible. It’s better for them if users (and, perhaps more pointedly, regulators) remain largely unaware of the constant negotiations and often fraught decisions they must make behind the scenes to maintain an image as neutral facilitators of content rather than being seen as a new kind of editorial power.

That moderation has, despite companies’ best efforts, found its way into broader public and regulatory conversations is, as Gillespie suggests, evidence that we’re seeing “social media’s slow and bumpy maturation, its gathering recognition that it is a powerful infrastructure for knowledge, participation, and public expression.” In view of this, he asks us to imagine a world in which social media platforms not only claim the right to moderate but also fully absorb “responsibility for their role in organizing, curating, and profiting from the activity of their users.”

Even if unrealistic, it’s an earned conclusion. It resonates not despite but precisely because it is backed by decades of online history, scholarship, and industry development. In that regard, the book is a notable departure from other entries in the genre, such as 2011’s The Filter Bubble or 2001’s Republic 2.0, that have been, at times, heavy on political prognostication but light on social scientific substance.

For scholars, activists, and practitioners already steeped in the politics of online platforms, Custodians of the Internet offers both a comprehensive retrospective and critical provocation. The book effectively documents where we’ve been, where we are now, and where—barring the sorts of changes Gillespie suggests—we’re headed.

But this book isn’t only for those already steeped in the practices of content moderation; it’s also for audiences who have only recently become aware of how deeply filtering, policy, and human judgment shape everyday life online. For newcomers, Gillespie offers an accessible and wide-ranging introduction to the dirty work of cleaning up the internet.

About the author

The reviewer is at the Information School, University of Washington, Seattle, WA 98195, USA.